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voluntary  provision  of  that  type  of  information"  (15  USC  7306c). 

In  addition,  a  substantial  portion  of  the  evidence  collected  by  NIST  in  the  course  of  the  investigation  has  been 
provided  to  NIST  under  nondisclosure  agreements. 


Disclaimer  No.  4 

NIST  takes  no  position  as  to  whether  the  design  or  construction  of  a  WTC  building  was  compliant  with  any  code 
since,  due  to  the  destruction  of  the  WTC  buildings,  NIST  could  not  verify  the  actual  (or  as-built)  construction,  the 
properties  and  condition  of  the  materials  used,  or  changes  to  the  original  construction  made  over  the  life  of  the 
buildings.  In  addition,  NIST  could  not  verify  the  interpretations  of  codes  used  by  applicable  authorities  in  determining 
compliance  when  implementing  building  codes.  Where  an  Investigation  report  states  whether  a  system  was 
designed  or  installed  as  required  by  a  code  provision,  NIST  has  documentary  or  anecdotal  evidence  indicating 
whether  the  requirement  was  met,  or  NIST  has  independently  conducted  tests  or  analyses  indicating  whether  the 
requirement  was  met. 


Use  in  Legal  Proceedings 

No  part  of  any  report  resulting  from  a  NIST  investigation  into  a  structural  failure  or  from  an  investigation  under  the 
National  Construction  Safety  Team  Act  may  be  used  in  any  suit  or  action  for  damages  arising  out  of  any  matter 
mentioned  in  such  report  (15  USC  281a;  as  amended  by  P.L.  107-231). 


National  Institute  of  Standards  and  Technology  National  Construction  Safety  Team  Act  Report  1-7B 
Natl.  Inst.  Stand.  Technol.  Natl.  Constr.  Sfty.  Tm.  Act  Rpt.  1-7B,  79  pages  (September  2005) 
CODEN:  NSPUE2 


U.S.  GOVERNMENT  PRINTING  OFFICE 
WASHINGTON:  2005 


For  sale  by  the  Superintendent  of  Documents,  U.S.  Government  Printing  Office 
Internet:  bookstore.gpo.gov  —  Phone:  (202)  512-1800  —  Fax:  (202)  512-2250 
Mail:  Stop  SSOP,  Washington,  DC  20402-0001 


Abstract 


This  report  documents  the  methods  and  outcomes  of  the  telephone  survey,  face-to-face  interviews,  and 
focus  groups  that  were  conducted  in  support  of  the  federal  building  and  fire  safety  investigation  of  the 
World  Trade  Center  disaster.  In  total,  803  telephone  interviews  were  completed,  220  face-to-face 
interviews  were  completed,  and  6  focus  groups  involving  28  individuals  were  completed.  All  interview 
material  was  transferred  to  National  Institute  of  Standards  and  Technology  investigators  for  analysis. 

Keywords:  Survey,  face-to-face  interviews,  focus  groups.  World  Trade  Center. 
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Preface 


Genesis  of  This  Investigation 

Immediately  following  the  terrorist  attack  on  the  World  Trade  Center  (WTC)  on  September  1 1 ,  2001 ,  the 
Federal  Emergency  Management  Agency  (FEMA)  and  the  American  Society  of  Civil  Engineers  began 
planning  a  building  performance  study  of  the  disaster.  The  week  of  October  7,  as  soon  as  the  rescue  and 
search  efforts  ceased,  the  Building  Performance  Study  Team  went  to  the  site  and  began  its  assessment. 
This  was  to  be  a  brief  effort,  as  the  study  team  consisted  of  experts  who  largely  volunteered  their  time 
away  from  their  other  professional  commitments.  The  Building  Performance  Study  Team  issued  its 
report  in  May  2002,  fulfilling  its  goal  "to  determine  probable  failure  mechanisms  and  to  identify  areas  of 
future  investigation  that  could  lead  to  practical  measures  for  improving  the  damage  resistance  of  buildings 
against  such  unforeseen  events." 

On  August  21,  2002,  with  funding  from  the  U.S.  Congress  through  FEMA,  the  National  Institute  of 
Standards  and  Technology  (NIST)  announced  its  building  and  fire  safety  investigation  of  the  WTC 
disaster.  On  October  1,  2002,  the  National  Constmction  Safety  Team  Act  (Public  Law  107-231),  was 
signed  into  law.  The  NIST  WTC  Investigation  was  conducted  under  the  authority  of  the  National 
Construction  Safety  Team  Act. 

The  goals  of  the  investigation  of  the  WTC  disaster  were: 

•  To  investigate  the  building  construction,  the  materials  used,  and  the  technical  conditions  that 
contributed  to  the  outcome  of  the  WTC  disaster. 

•  To  serve  as  the  basis  for: 

-  Improvements  in  the  way  buildings  are  designed,  constructed,  maintained,  and  used; 

-  Improved  tools  and  guidance  for  industry  and  safety  officials; 

-  Recommended  revisions  to  current  codes,  standards,  and  practices;  and 

-  Improved  public  safety. 

The  specific  objectives  were: 

1.  Determine  why  and  how  WTC  1  and  WTC  2  collapsed  following  the  initial  impacts  of  the 
aircraft  and  why  and  how  WTC  7  collapsed; 

2.  Determine  why  the  injuries  and  fatalities  were  so  high  or  low  depending  on  location, 
including  all  technical  aspects  of  fire  protection,  occupant  behavior,  evacuation,  and 
emergency  response; 

3.  Determine  what  procedures  and  practices  were  used  in  the  design,  construction,  operation, 
and  maintenance  of  WTC  1,  2,  and  7;  and 

4.  Identify,  as  specifically  as  possible,  areas  in  current  building  and  fire  codes,  standards,  and 
practices  that  warrant  revision. 
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NIST  is  a  nonregulatory  agency  of  the  U.S.  Department  of  Commerce's  Technology  Administration.  The 
purpose  of  NIST  investigations  is  to  improve  the  safety  and  structural  integrity  of  buildings  in  the  United 
States,  and  the  focus  is  on  fact  finding.  NIST  investigative  teams  are  authorized  to  assess  building 
perfonnance  and  emergency  response  and  evacuation  procedures  in  the  wake  of  any  building  failure  that 
has  resulted  in  substantial  loss  of  life  or  that  posed  significant  potential  of  substantial  loss  of  life.  NIST 
does  not  have  the  statutory  authority  to  make  findings  of  fault  nor  negligence  by  individuals  or 
organizations.  Further,  no  part  of  any  report  resulting  from  a  NIST  investigation  into  a  building  failure  or 
from  an  investigation  under  the  National  Construction  Safety  Team  Act  may  be  used  in  any  suit  or  action 
for  damages  arising  out  of  any  matter  mentioned  in  such  report  (15  USC  281a,  as  amended  by  Public 
Law  107-231). 

Organization  of  the  Investigation 

The  National  Construction  Safety  Team  for  this  Investigation,  appointed  by  the  then  NIST  Director, 
Dr.  Arden  L.  Bement,  Jr.,  was  led  by  Dr.  S.  Shyam  Sunder.  Dr.  William  L.  Grosshandler  served  as 
Associate  Lead  Investigator,  Mr.  Stephen  A.  Cauffman  served  as  Program  Manager  for  Administration, 
and  Mr.  Harold  E.  Nelson  served  on  the  team  as  a  private  sector  expert.  The  Investigation  included  eight 
interdependent  projects  whose  leaders  comprised  the  remainder  of  the  team.  A  detailed  description  of 
each  of  these  eight  projects  is  available  at  http://wtc.nist.gov.  The  purpose  of  each  project  is  summarized 
in  Table  P-1,  and  the  key  interdependencies  among  the  projects  are  illustrated  in  Fig.  P-1. 


Table  P-1.  Federal  building  and  fire  safety  investigation  of  the  WTC  disaster. 


Technical  Area  and  Project  Leader 

Project  Purpose 

Analysis  of  Building  and  Fire  Codes  and 
Practices;  Project  Leaders:  Dr.  H.  S.  Lew 
and  Mr.  Richard  W.  Bukowski 

Document  and  analyze  the  code  provisions,  procedures,  and 
practices  used  in  the  design,  construction,  operation,  and 
maintenance  of  the  structural,  passive  fire  protection,  and 
emergency  access  and  evacuation  systems  of  WTC  1,  2,  and  7. 

Baseline  Structural  Performance  and 
Aircraft  hnpact  Damage  Analysis;  Project 
Leader:  Dr.  Fahim  H.  Sadek 

Analyze  the  baseline  performance  of  WTC  1  and  WTC  2  under 
design,  service,  and  abnonnal  loads,  and  aircraft  impact  damage  on 
the  structural,  fire  protection,  and  egress  systems. 

Mechanical  and  Metallurgical  Analysis  of 
Structural  Steel;  Project  Leader:  Dr.  Frank 
W.  Gayle 

Determine  and  analyze  the  mechanical  and  metallurgical  properties 
and  quality  of  steel,  weldments,  and  connections  from  steel 
recovered  from  WTC  1,  2,  and  7. 

Investigation  of  Active  Fire  Protection 
Systems;  Project  Leader:  Dr.  David 
D.  Evans;  Dr.  William  Grosshandler 

Investigate  the  performance  of  the  active  fire  protection  systems  in 
WTC  1,2,  and  7  and  their  role  in  fire  control,  emergency  response, 
and  fate  of  occupants  and  responders. 

Reconstruction  of  Thermal  and  Tenability 
Environment;  Project  Leader:  Dr.  Richard 
G.  Gann 

Reconstruct  the  time-evolving  temperature,  thermal  environment, 
and  smoke  movement  in  WTC  1,  2,  and  7  for  use  in  evaluating  the 
structural  performance  of  the  buildings  and  behavior  and  fate  of 
occupants  and  responders. 

Structural  Fire  Response  and  Collapse 
Analysis;  Project  Leaders:  Dr.  John 
L.  Gross  and  Dr.  Therese  P.  McAllister 

Analyze  the  response  of  the  WTC  towers  to  fires  with  and  without 
aircraft  damage,  the  response  of  WTC  7  in  fires,  the  perfonnance 
of  composite  steel-trussed  floor  systems,  and  deteraiine  the  most 
probable  structural  collapse  sequence  for  WTC  1,  2.  and  7. 

Occupant  Behavior,  Egress,  and  Emergency 
Communications;  Project  Leader:  Mr.  Jason 
D.  Averill 

Analyze  the  behavior  and  fate  of  occupants  and  responders,  both 
those  who  survived  and  those  who  did  not,  and  the  performance  of 
the  evacuation  system. 

Emergency  Response  Technologies  and 
Guidelines;  Project  Leader:  Mr.  J.  Randall 
Lawson 

Document  the  activities  of  the  emergency  responders  from  the  time 
of  the  terrorist  attacks  on  WTC  1  and  WTC  2  until  the  collapse  of 
WTC  7,  including  practices  followed  and  technologies  used. 
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Figure  P-1.  The  eight  projects  in  the  federal  building  and  fire  safety 
investigation  of  the  WTC  disaster. 


National  Construction  Safety  Team  Advisory  Committee 

The  NIST  Director  also  established  an  advisory  committee  as  mandated  under  the  National  Construction 
Safety  Team  Act.  The  initial  members  of  the  committee  were  appointed  following  a  public  solicitation. 
These  were: 

•    Paul  Fitzgerald,  Executive  Vice  President  (retired)  FM  Global,  National  Construction  Safety 
Team  Advisory  Committee  Chair 


•  John  Barsom,  President,  Barsom  Consulting,  Ltd. 

•  John  Bryan,  Professor  Emeritus,  University  of  Maryland 

•  David  Collins,  President,  The  Preview  Group,  Inc. 

•  Glerm  Corbett,  Professor,  John  Jay  College  of  Criminal  Justice 

•  Philip  DiNenno,  President,  Hughes  Associates,  Inc. 
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•  Robert  Hanson,  Professor  Emeritus,  University  of  Michigan 

•  Charles  Thornton,  Co-Chainnan  and  Managing  Principal,  The  Thomton-Tomasetti  Group, 
Inc. 

•  Kathleen  Tiemey,  Director,  Natural  Hazards  Research  and  Applications  hiformation  Center, 
University  of  Colorado  at  Boulder 

•  Forman  Williams,  Director,  Center  for  Energy  Research,  University  of  California  at  San 
Diego 

This  National  Construction  Safety  Team  Advisory  Committee  provided  technical  advice  during  the 
Investigation  and  commentary  on  drafts  of  the  Investigation  reports  prior  to  their  public  release.  NIST 
has  benefited  from  the  work  of  many  people  in  the  preparation  of  these  reports,  including  the  National 
Construction  Safety  Team  Advisory  Committee.  The  content  of  the  reports  and  recommendations, 
however,  are  solely  the  responsibility  of  NIST. 

Public  Outreach 

During  the  course  of  this  Investigation,  NIST  held  public  briefings  and  meetings  (listed  in  Table  P-2)  to 
solicit  input  from  the  public,  present  preliminary  findings,  and  obtain  comments  on  the  direction  and 
progress  of  the  Investigation  from  the  public  and  the  Advisory  Committee. 

NIST  maintained  a  publicly  accessible  Web  site  during  this  Investigation  at  http://wtc.nist.gov.  The  site 
contained  extensive  information  on  the  background  and  progress  of  the  Investigation. 

NIST's  WTC  Public-Private  Response  Plan 

The  collapse  of  the  WTC  buildings  has  led  to  broad  reexamination  of  how  tall  buildings  are  designed, 
constructed,  maintained,  and  used,  especially  with  regard  to  major  events  such  as  fires,  natural  disasters, 
and  terrorist  attacks.  Reflecting  the  enhanced  interest  in  effecting  necessary  change,  NIST,  with  support 
from  Congress  and  the  Administration,  has  put  in  place  a  program,  the  goal  of  which  is  to  develop  and 
implement  the  standards,  technology,  and  practices  needed  for  cost-effective  improvements  to  the  safety 
and  security  of  buildings  and  building  occupants,  including  evacuation,  emergency  response  procedures, 
and  threat  mitigation. 

The  strategy  to  meet  this  goal  is  a  three-part  NIST-led  public-private  response  program  that  includes: 

•  A  federal  building  and  fire  safety  investigation  to  study  the  most  probable  factors  that 
contributed  to  post-aircraft  impact  collapse  of  the  WTC  towers  and  the  47-story  WTC  7 
building,  and  the  associated  evacuation  and  emergency  response  experience. 

•  A  research  and  development  (R&D)  program  to  (a)  facilitate  the  implementation  of 
recommendations  resulting  from  the  WTC  Investigation,  and  (b)  provide  the  technical  basis 
for  cost-effective  improvements  to  national  building  and  fire  codes,  standards,  and  practices 
that  enhance  the  safety  of  buildings,  their  occupants,  and  emergency  responders. 
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Table  P-2.  Public  meetings  and  briefings  of  the  WTC  Investigation. 

Date 

Location 

Principal  Agenda 

June  24, 2002 

New  York  City,  NY 

Public  meeting:  Public  comments  on  the  Draf!  Plan  for  the 
pending  WTC  Investigation. 

August  21.  2002 

Gaithersburg,  MD 

Media  briefing  announcing  the  formal  start  of  the  Investigation. 

December  9.  2002 

Washington,  DC 

Media  briefing  on  release  of  the  Public  Update  and  NIST  request 
for  photographs  and  videos. 

April  8,  2003 

New  York  City,  NY 

Joint  public  forum  with  Columbia  University  on  first-person 
interviews. 

April  29-30,  2003 

Gaithersburg,  MD 

NCST  Advisory  Committee  meeting  on  plan  for  and  progress  on 
WTC  Investigation  with  a  public  comment  session. 

May  7,  2003 

New  York  City,  NY 

Media  briefing  on  release  of  May  2003  Progress  Report. 

August  26-27,  2003 

Gaithersburg,  MD 

NCST  Advisory  Committee  meeting  on  status  of  the  WTC 
investigation  with  a  public  comment  session. 

September  17,  2003 

New  York  City.  NY 

Media  and  public  briefing  on  initiation  of  first-person  data 
collection  projects. 

December  2-3.  2003 

Gaithersburg,  MD 

NCST  Advisory  Committee  meeting  on  status  and  initial  results 
and  release  of  the  Public  Update  with  a  public  comment  session. 

February  12,  2004 

New  York  City,  NY 

Public  meetmg  on  progress  and  preliminary  findings  with  public 
comments  on  issues  to  be  considered  in  formulating  final 
recommendations. 

June  18. 2004 

New  York  City,  NY 

Media/public  briefing  on  release  of  June  2004  Progress  Report. 

June  22-23,  2004 

Gaithersburg,  MD 

NCST  Advisory  Committee  meeting  on  the  status  of  and 
preliminary  findings  from  the  WTC  Investigation  with  a  public 
comment  session. 

August  24,  2004 

Northbrook,  IL 

Public  viewing  of  standard  fire  resistance  test  of  WTC  floor 
system  at  Underwriters  Laboratories,  Inc. 

October  19-20,  2004 

Gaithersburg,  MD 

NCST  Advisory  Committee  meeting  on  status  and  near  complete 
set  of  preliminary  findings  with  a  public  comment  session. 

iNovemoer  J.z,  /uu4 

Gaithersburg,  MD 

iNt^oi  /VQVisory  ^^ommiuee  uiscussion  on  uraii  dimudi  icpoii  lo 
Congress,  a  public  comment  session,  and  a  closed  session  to 
discuss  pre-draft  recommendations  for  WTC  Investigation. 

April  5,  2005 

New  York  City,  NY 

Media  and  public  briefing  on  release  of  the  probable  collapse 
sequence  for  the  WTC  towers  and  draft  reports  for  the  projects  on 
codes  and  practices,  evacuation,  and  emergency  response. 

June  23, 2005 

New  York  City,  NY 

Media  and  public  briefing  on  release  of  all  draft  reports  for  the 
WTC  towers  and  draft  recommendations  for  public  comment. 

September  12-13, 
2005 

Gaithersburg,  MD 

NCST  Advisory  Committee  meeting  on  disposition  of  public 
comments  and  update  to  draft  reports  for  the  WTC  towers. 

September  13-15, 
2005 

Gaithersburg,  MD 

WTC  Technical  Conference  for  stakeholders  and  technical 
community  for  dissemination  of  findings  and  recommendations 
and  opportunity  for  public  to  make  technical  comments. 

•    A  dissemination  and  technical  assistance  program  (DTAP)  to  (a)  engage  leaders  of  the 
construction  and  building  community  in  ensuring  timely  adoption  and  widespread  use  of 
proposed  changes  to  practices,  standards,  and  codes  resuhing  from  the  WTC  Investigation 
and  the  R&D  program,  and  (b)  provide  practical  guidance  and  tools  to  better  prepare  faciUty 
owners,  contractors,  architects,  engineers,  emergency  responders,  and  regulatory  authorities 
to  respond  to  future  disasters. 

The  desired  outcomes  are  to  make  buildings,  occupants,  and  first  responders  safer  in  future  disaster 
events. 


NISTNCSTAR  1-7B,  WTC  Investigation 


xvii 


Preface 


National  Construction  Safety  Team  Reports  on  the  WTC  Investigation 

A  final  report  on  the  collapse  of  the  WTC  towers  is  being  issued  as  NIST  NCSTAR  1 .  A  companion 
report  on  the  collapse  of  WTC  7  is  being  issued  as  NIST  NCSTAR  I  A.  The  present  report  is  one  of  a  set 
that  provides  more  detailed  documentation  of  the  Investigation  findings  and  the  means  by  which  these 
technical  results  were  achieved.  As  such,  it  is  part  of  the  archival  record  of  this  Investigation.  The  titles 
of  the  full  set  of  Investigation  publications  are: 
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NIST  (National  Institute  of  Standards  and  Technology).  2006.  Federal  Building  and  Fire  Safety 
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NIST  NCSTAR  1  A.  Gaithersburg,  MD. 

Lew,  H.  S.,  R.  W.  Bukowski,  and  N.  J.  Carino.  2005.  Federal  Building  and  Fire  Safety  Investigation  of 
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Systems.  NIST  NCSTAR  1-1.  National  Institute  of  Standards  and  Technology.  Gaithersburg,  MD, 
September. 

Fanella,  D.  A.,  A.  T.  Derecho,  and  S.  K.  Ghosh.  2005.  Federal  Building  and  Fire  Safety 
Investigation  of  the  World  Trade  Center  Disaster:  Design  and  Construction  of  Structural  Systems. 
NIST  NCSTAR  1-1  A.  National  Institute  of  Standards  and  Technology.  Gaithersburg,  MD, 
September. 

Ghosh,  S.  K.,  and  X.  Liang.  2005.  Federal  Building  and  Fire  Safety  Investigation  of  the  World 
Trade  Center  Disaster:  Comparison  of  Building  Code  Structural  Requirements.  NIST 
NCSTAR  1-lB.  National  Institute  of  Standards  and  Technology.  Gaithersburg,  MD,  September. 

Fanella,  D.  A.,  A.  T.  Derecho,  and  S.  K.  Ghosh.  2005.  Federal  Building  and  Fire  Safety 
Investigation  of  the  World  Trade  Center  Disaster:  Maintenance  and  Modifications  to  Structural 
Systems.  NIST  NCSTAR  1-lC.  National  Institute  of  Standards  and  Technology.  Gaithersburg, 
MD,  September. 

Grill,  R.  A.,  and  D.  A.  Johnson.  2005.  Federal  Building  and  Fire  Safety  Investigation  of  the  World 
Trade  Center  Disaster:  Fire  Protection  and  Life  Safety  Provisions  Applied  to  the  Design  and 
Construction  of  World  Trade  Center  7,  2.  and  7  and  Post-Construction  Provisions  Applied  after 
Occupancy.  NIST  NCSTAR  1- ID.  National  Institute  of  Standards  and  Technology.  Gaithersburg, 
MD,  September.  ,  ■ 

Razza,  J.  C,  and  R.  A.  Grill.  2005.  Federal  Building  and  Fire  Safety  Investigation  of  the  World 
Trade  Center  Disaster:  Comparison  of  Codes,  Standards,  and  Practices  in  Use  at  the  Time  of  the 
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York  City  Building  Code  Provisions.  NIST  NCSTAR  1-lF.  National  Institute  of  Standards  and 
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Executive  Summary 


Survey  administration  involved  interviews  with  survivors  of  the  World  Trade  Center  (WTC)  attacks  using 
three  methods,  telephone  interviews,  face-to-face  interviews,  and  focus  groups,  and  the  delivery  of  the 
resuhant  data.  This  report  documents  the  methods  and  outcomes  of  these  interviews.  Analyses  of  the 
resuhant  data  were  conducted  by  National  Institute  of  Standards  and  Technology  (NIST)  investigators. 
The  interviews  were  carried  out  by  a  data  collection  team  consisting  of  NuStats  Partners,  LP,  Austin, 
Texas,  as  the  prime  contractor,  with  the  assistance  of  subcontractors  -  NuStats  DataSource,  San  Marcos, 
Texas,  to  conduct  telephone  interviews;  GeoStats,  Atlanta,  Georgia,  to  provide  data  collection  devices 
and  programming  services  for  the  face-to-face  interviews;  and  Diversity  Services,  New  York  City,  to 
recruit  surveyors  and  provide  office  space. 

The  goals  of  the  NIST  World  Trade  Center  investigation  were  to  investigate  the  building  construction,  the 
materials  used,  and  the  technical  conditions  that  contributed  to  the  outcome  of  the  WTC  disaster.  The 
results  of  the  investigation  were  meant  to  serve  as  the  basis  for  improvements  in  the  way  buildings  are 
designed,  constructed,  maintained,  and  used;  improved  tools,  guidance  for  industry  and  safety  officials; 
revisions  to  codes,  standards,  and  practices;  and  improved  public  safety. 

This  research  was  conducted  in  compliance  with  NIST  requirements  regarding  the  protection  of  human 
subjects.  The  Office  of  Human  Research  Protections  (OHRP)  in  the  Department  of  Health  and  Human 
Services  (DHHS)  registered  Federalwide  Assurance  (FWA)  numbers  for  both  NuStats  Partners,  LP  and 
NuStats  DataSource,  LP.  For  both  of  these  FWA  entities,  the  Institutional  Review  Board  (IRB) 
designated  for  review  was  Essex  IRB,  Lebanon,  New  Jersey 

NuStats  prepared  a  comprehensive  IRB  package  for  all  data  collection  protocols,  and  submitted  the 
package  to  Essex  IRB.  Subsequent  to  NIST  review,  Essex  approved  a  revised  IRB  package.  An 
important  partner  in  the  provision  of  human  subjects  protection  was  Jamie  Abelson,  a  senior  research 
associate  affiliated  with  the  University  of  Michigan  Institute  for  Social  Research.  Based  on  her  vast 
experience  in  dealing  with  sensitive  interviewing  situations,  she  provided  hands-on  mental  health  support, 
as  needed,  to  both  surveyors  and  respondents.  NIST  further  worked  with  NuStats  to  ensure  that  the  data 
collection  met  the  requirements  of  the  Paperwork  Reduction  Act  and  was  awarded  an  Office  of 
Management  and  Budget  (0MB)  approval  number  (0693-0044). 

Three  data  collection  methodologies  were  employed  for  this  investigation,  including:  (1)  a  telephone 
interview  of  803  occupants  of  the  WTC  towers,  utilizing  a  scientific  probability  sample  of  sur\dving 
occupants  was  used  to  identify  survey  participants;  (2)  a  qualitative  study  of  220  face-to-face  interviews 
of  occupants  and  family  members  of  victims,  utilizing  a  snowball  quota  sample  of  key  survivor  types  and 
the  toll-free  hotline  were  used  to  identify  potential  participants;  and  (3)  six  focus  groups  of  occupants, 
relying  on  a  snowball  quota  sampling  methodology  for  recruiting  key  survivor  participants. 
The  primary  goal  of  the  telephone  interview  was  to  provide  quantitative  information  on  occupant 
behavior  during  the  evacuation  experience  and  capture  unique  investigative  observ^ations,  particular  to  the 
events  at  the  WTC  on  September  11,  2001.  This  data  collection  activity  also  sought  to  estimate  the 
occupant  population  in  the  WTC  at  the  time  of  the  attacks  on  September  1 1 .  The  mode  of  data  collection 
was  computer-assisted  telephone  interviewing  (CATI). 
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The  interview  was  designed  to  measure  five  primary  content  areas,  presented  to  the  respondent  in  the 
following  order: 

1.  Preparedness  and  Training  -  These  questions  collected  information  on  the  respondents' 
degree  of  evacuation  or  emergency  training  pertinent  to  their  September  1 1  experience.  The 
questions  also  addressed  famiharity  with  the  WTC  buildings  based  on  length  employment  at 
the  location  and  also  whether  the  respondents  were  at  the  WTC  in  1993  during  that  prior 
bombing  incident. 

2.  Initial  September  1 1  Experience  -  These  questions  captured  data  on  respondents'  initial 
awareness  that  something  had  happened  to  the  building  -  where  they  were,  what  they  were 
doing,  how  they  first  became  aware,  sources  of  information  about  the  event  (both  in  terms  of 
other  persons  and  also  in  terms  of  their  physical  environment). 

3.  Interim  September  1 1  Experience  -  These  questions  addressed  the  span  between  the  time 
respondents  first  became  aware  of  the  WTC  incident  and  when  they  began  evacuating  -  data 
items  included  their  social  environment,  physical  environment,  information  sources,  and 
factors  influencing  their  decision  to  evacuate. 

4.  Evacuation  Experience  on  September  1 1  -  These  questions  captured  information  on  the 
respondents'  egress  from  the  building,  including  the  factors  that  facilitated  their  egress  and 
those  that  hindered  their  egress. 

5.  Respondent  Characteristics  -  Information  collected  included  age,  gender,  language,  and 
mobility  impairments. 

The  objective  of  the  face-to-face  interviews  was  to  gather  first-hand  accounts  and  observations  of  the 
activities  and  events  inside  the  WTC  buildings  on  the  morning  of  September  1 1 .  The  qualitative  nature  of 
the  face-to-face  interviews  identified  heretofore  unknown  information,  evaluated  technical  hypotheses, 
and  explored  conscious  and  subconscious  motivations  for  occupant  and  responder  behaviors,  while 
allowing  for  qualitative  comparisons  to  the  telephone  interview  data.  The  face-to-face  interview 
methodology  was  a  synthesis  of  the  Behavioral  Sequence  Interview  Technique  originally  developed  by 
Keating  and  Loftus,  and  the  Cognitive  Interviewing  Methods,  originally  developed  by  Fisher  and 
Gieselman. 

The  following  special  groups  of  building  occupants  were  targeted  in  the  face-to-face  interviews.  NIST 
and  the  expert  panel  selected  these  special  groups  for  the  face-to-face  interviews  because  these  groups 
were  believed  to  hold  unique  pieces  of  information  to  add  to  the  investigation.  They  either  belong  to 
special  subpopulation  groups  or  were  in  locations  during  the  evacuation  that  enabled  them  to  experience 
distinctive  situations.  Because  these  respondents  were  expected  to  represent  relatively  rare  types  of 
occupants  or  other  targeted  groups,  a  snowball  sampling  approach  was  used  to  locate  respondents. 
Snowball  sampling  relies  on  referrals  from  initial  subjects  to  generate  additional  subjects.  The  selection 
of  this  approach,  a  non-probability  design,  eliminated  the  potential  for  generalizing  to  the  population  of 
inference  (i.e.,  occupants  of  WTC  towers  on  September  11),  but  the  design  provided  access  to  rare,  hard- 
to-find  persons  with  unique  information  to  provide  to  the  investigation.  The  special  groups  targeted  in  the 
face-to-face  interviews  were:  (a)  people  near  the  floors  of  impact;  (b)  people  in  the  lobby  who  witnessed 
fireballs;  (c)  families  of  victims  who  talked  to  victim  during  disaster;  (d)  Occupants  from  WTC  7;  (e) 
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persons  with  building  responsibilities;  (f)  people  who  used  or  were  trapped  in  elevators;  (g)  people  with 
mobility  challenges;  and  (h)  random  evaeuees  in  WTC  1  and  WTC  2. 

The  face-to-face  interview  instrument  was  drafted  as  a  computer-assisted  personal  interview  (CAPl) 
application  that  would  operate  on  Pen  Tablet  PCs.  With  CAPI  technology,  high  quality  qualitative  data 
were  electronically  coded,  organized,  and  stored  directly  in  an  electronic  fonnat,  with  all  relevant  error 
cheeking  performed  automatically.  Not  only  was  this  application  more  appropriate  for  the  prescribed  data 
collection  protocol,  this  process  simultaneously  eliminated  unnecessary  data  entry  costs  and  associated 
key  entry  errors  that  typically  occur  in  paper-based  field  surveys.  Both  the  Pen  Tablet  PCs  and  the 
application  were  provided  by  GeoStats. 

The  goal  of  the  focus  groups  was  to  elicit  group  representations  of  specific  events  or  themes.  Distinct 
categories  of  persons  were  provided  by  NIST  based  on  preliminary  analyses  of  CATI  survey  and  face-to- 
face  interview  data.  These  groups  were  prioritized  based  on  expectations  of  sample  availability  and 
information  value. 

Six  focus  groups  were  completed.  The  groups  were: 

1 .  Random  survivors  from  WTC  1 ; 

2.  Random  survivors  from  WTC  2; 

3.  Mobihty  challenged  individuals; 

4.  Persons  having  building  responsibility; 

5.  Persons  near  the  floors  of  impact;  and 

6.  Floor  wardens. 

The  venues  for  the  focus  groups  were  changed  from  Manhattan  to  outside  of  Manhattan  depending  on  the 
preferences  of  focus  group  respondents.  Among  respondents  who  were  willing  to  participate  in  the  focus 
group,  several  no  longer  traveled  into  Manhattan.  Of  the  persons  who  participated  in  the  groups,  all  were 
active  in  the  discussion  and  willing  to  share  their  experiences,  thoughts,  and  recommendations. 

NuStats  was  responsible  for  delivery  of  databases  containing  information  resulting  from  the  interviews 
with  survivors.  These  databases  comprised  various  formats,  specific  to  the  type  of  interview  conducted. 

In  addition  to  delivering  databases,  NuStats  also  provided  database  support  services.  These  services 
included  locating  a  subcontractor  to  provide  qualitative  analysis  support  for  the  faee-to-face  interviews 
and  providing  statistical  expertise  to  support  model  development  using  the  CATI  data. 
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Chapter  1 
Introduction 


Survey  administration  involved  interviews  with  survivors  of  the  World  Trade  Center  (WTC)  attacks  using 
three  methods,  telephone  interviews,  face-to-face  interviews,  and  focus  groups,  and  the  delivery  of  the 
resultant  data.  This  report  documents  the  methods  and  outcomes  of  these  interviews.  (Appendix  A 
contains  the  work  program  that  guided  the  work  conducted  herein.')  Analyses  of  the  resultant  data  were 
conducted  by  National  Institute  of  Standards  and  Technology  (NIST)  investigators.  The  interviews  were 
carried  out  by  a  data  collection  team  consisting  of  NuStats  Partners,  LP,  Austin,  Texas,  as  the  prime 
contractor,  with  the  assistance  of  subcontractors — ^NuStats  DataSource,  San  Marcos,  Texas,  to  conduct 
telephone  interviews;  GeoStats,  Atlanta,  Georgia,  to  provide  data  collection  devices  and  programming 
services  for  the  face-to-face  interviews;  and  Diversity  Services,  New  York  City,  to  recruit  surveyors  and 
provide  office  space. 


1.1  OBJECTIVE 


The  goals  of  the  NIST  WTC  investigation  were  to  investigate  the  building  construction,  the  materials 
used,  and  the  technical  conditions  that  contributed  to  the  outcome  of  the  WTC  disaster.  The  results  of  the 
investigation  were  meant  to  serve  as  the  basis  for  improvements  in  the  way  buildings  are  designed, 
constructed,  maintained,  and  used;  improved  tools,  guidance  for  industry  and  safety  officials;  revisions  to 
codes,  standards,  and  practices;  and  improved  public  safety.  While  multiple  sources  and  types  of  data 
were  sought  to  meet  the  needs  of  the  NIST  investigation,  the  scope  of  services  under  this  contract 
provided  important  first-hand  accounts  of  events  and  experiences  on  September  1 1 ,  200 1 ,  through 
telephone  interviews,  face-to-face  interviews,  and  focus  groups.  This  data  collection  effort  evaluated  the 
role  of  occupant  behavior  and  evacuation  technologies  and  practices  for  tall  buildings,  including  decision- 
making and  situation  awareness,  time-constrained  evacuation  strategies,  communications,  role  of  floor 
wardens  and  fire  safety  directors,  and  issues  concerning  people  with  disabilities.  Additionally, 
observations  of  fire  and  smoke  conditions  and/or  structural  damage  from  within  the  building  were  sought. 
Families  of  the  victims,  or  others  who  communicated  with  loves  ones  inside  the  towers  before  collapse, 
were  also  interviewed  primarily  to  determine  the  nature  of  the  environment  above  the  floors  of  impact. 


1 .2     PROTECTION  OF  HUMAN  SUBJECTS 


This  research  was  conducted  in  comphance  with  NIST  requirements  regarding  the  protection  of  human 
subjects.  The  Office  of  Human  Research  Protections  (OHRP)  in  the  Department  of  Health  and  Human 
Services  (DHHS)  registered  Federalwide  Assurance  (FWA)  numbers  for  both  NuStats  Partners,  LP 
(FWA00000562)  and  NuStats  DataSource,  LP  (FWA00005084).  For  both  of  these  FWA  entities,  the 
Institutional  Review  Board  (IRB)  designated  for  review  was  Essex  IRB,  Lebanon,  New  Jersey.  Essex 


'  Whereas  the  work  program  provided  in  Appendix  A  refers  to  interviews  with  first  responders,  subsequent  agreements  between 
NIST  and  the  City  of  New  York  led  to  NIST  personnel  conducting  these  interviews.  This  level  of  effort  by  the  contractor  was 
replaced  by  the  provision  of  other  services. 
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IRB  is  registered  with  OHRP  and  is  in  good  standing  with  OHRP.  Essex's  IRB  identification  number  is 
IRB00001742. 

NuStats  prepared  a  comprehensive  IRB  package  for  all  data  collection  protocols,  and  submitted  the 
package  to  Essex  IRB  in  September  2003.  Essex  approved  this  package  on  September  26.  In 
October  2003,  NuStats  revised  the  IRB  package  based  on  NIST  legal  review,  and  then  NuStats  submitted 
the  revised  IRB  package  to  Essex  IRB  for  review.  On  November  10,  Essex  approved  this  revised  IRB 
package.  Based  on  subsequent  issues  raised  by  NIST  legal,  NuStats  revised  the  IRB  package  once  again, 
and  submitted  this  revised  package  to  Essex  IRB  for  review.  On  November  17,  Essex  approved  this 
revised  IRB  package. 

An  important  partner  in  the  provision  of  human  subjects  protection  was  Jamie  Abelson,  a  senior  research 
associate  affiliated  with  the  University  of  Michigan  Institute  for  Social  Research.  Based  on  her  vast 
experience  in  dealing  with  sensitive  interviewing  situations,  she  provided  hands-on  mental  health  support, 
as  needed,  to  both  surveyors  and  respondents. 


The  Paperwork  Reduction  Act  of  1995,  effective  October  1,  1995,  requires  that  Office  of  Management 
and  Budget  (0MB)  approve  each  collection  of  information  by  a  Federal  agency  before  it  can  be 
implemented.  NIST  prepared  a  package  of  materials  for  the  telephone  data  collection.  NuStats  reviewed 
these  0MB  materials  to  ensure  consistency  with  survey  and  sample  design  parameters  and  with  0MB 
reporting  requirements.  In  September  2003,  NIST  was  granted  0MB  approval  number  0693-0044. 


At  NIST's  request,  NuStats  set  up  a  toll-free  (1-800)  number  with  a  staff  of  telephone  operators.  This  line 
was  used  to  capture  requests  for  information  about  the  NIST  investigation,  as  well  as  to  screen  potential 
respondents  for  the  telephone,  face-to-face,  or  focus  group  interviews.  A  screener  interview  was 
conducted  with  hotline  callers  to  cull  those  who  fit  the  necessary  profile  for  unique  groups  to  be 
interviewed,  primarily  in  the  face-to-face  interviewing  effort.  Summaries  of  hotline  contacts  by  day  were 
prepared  for  NIST  review. 


1.3 


PAPERWORK  REDUCTION  ACT 


1.4 


TOLL-FREE  HOTLINE 
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Three  data  collection  methodologies  were  employed  for  this  investigation,  as  presented  in  the  bullets 
below. 


•  Telephone  Survey  :  a  telephone  interview  survey  of  800  occupants  of  the  World  Trade 
Center  (WTC)  towers;  a  scientific  probability  sample  of  surviving  occupants  was  used  to 
identify  survey  participants. 

•  In-Depth  Personal  Interviews .  a  qualitative  study  of  approximately  220  face-to-face 
interviews  of  occupants  and  family  members  of  victims;  a  snowball  quota  sample  of  key 
survivor  types  and  the  toll-free  hotline  were  used  to  identify  potential  participants. 

•  Focus  Groups:  five  focus  groups  of  occupants  were  conducted;  this  method  also  relied  on  a 
snowball  quota  sampling  methodology  for  recruiting  key  survivor  participants. 

Prior  to  the  NuStats  contract.  National  Institute  of  Standards  and  Technology  (NIST)  staff  and  contractor 
experts  selected  this  multi-method  approach  for  several  reasons."  First,  multiple  methods  would  increase 
confidence  in  the  conclusions  and  findings  when  more  than  one  method  arrived  at  the  same  conclusions,  a 
process  known  as  triangulation.  Second,  the  multiple  objectives  of  the  investigation  mandated 
complementary  approaches  to  accomplish  all  the  goals.  In  other  words  with  only  one  method,  it  would  be 
difficult  to  establish  a  scientific  foundation  for  general  findings  while  also  broadly  investigating  and 
establishing  new  facts  and  discovering  unique  events.  Finally,  concerns  associated  with  the  time  latency 
since  September  11,  2001,  suggested  the  use  of  different  approaches  and  techniques  in  order  to  increase 
memory  recall  and  accuracy.^  The  original  methodology  paper  is  included  in  Appendix  B. 


2.1      TELEPHONE  SURVEY"* 


The  primary  goal  of  the  telephone  interview  was  to  provide  quantitative  information  on  occupant 
behavior  during  the  evacuation  experience  and  capture  unique  investigative  observations,  particular  to  the 
events  at  the  WTC  on  September  11,  2001 .  This  data  collection  activity  also  sought  to  estimate  the 


'  The  NIST  project  staff  primarily  involved  in  this  effort  included  Jason  Averill,  Erica  Kuligowski,  Randy  Lawson,  Richard 
Peacock,  and  Paul  Reneke.  Contractor  experts  included  Dr.  Norman  Groner,  John  Jay  College  of  Criminal  Justice  in  New 
York  City,  Dr.  Dennis  Mileti,  Director  of  the  National  Hazards  Research  and  Applications  Information  Center  within  the 
Institute  of  Behavioral  Science  at  the  University  of  Colorado  at  Boulder,  and  Dr.  Guylene  Proulx,  Research  Officer  from  the 
Institute  for  Research  in  Construction  at  the  National  Research  Council  of  Canada. 

^  From  "Data  Collection  Methodology  for  World  Trade  Center  Evaluation  and  Emergency  Response:  Telephone  Interviews, 
Face-to-Face  Interviews,  Focus  Groups  and  Population  Sampling."  Jason  Averill,  Erica  Kuligowski.  Randy  Lawson,  Richard 
Peacock,  and  Paul  Reneke,  National  Institute  of  Standards  and  Technology,  Gaithersburg,  MD  and  Norman  Groner,  Dennis 
Mileti,  and  Guylene  Proulx. 

The  information  in  this  section  relies  on  the  content  of  NIST  NCSTAR  1-7,  prepared  by  NIST  from  information  provided  by 
NuStats. 
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occupant  population  in  the  WTC  at  the  time  of  the  attacks  on  September  1 1 .  The  mode  of  data  collection 
was  computer-assisted  telephone  interviewing  (CATI). 

The  contract  required  the  collection  of  800  CATI  interviews  with  persons  occupying  either  of  the  two 
WTC  towers  (WTC  1,  WTC  2)  at  the  time  of  the  terrorist  attacks  on  September  11,  2001.  Attempts  were 
made  to  equally  divide  the  respondents  between  WTC  1  and  WTC  2  occupants  (i.e.,  n  =  400  occupant 
intei'views  from  each  tower).  A  proportionate  stratified  sample  design  drew  samples  of  survivors  within 
each  of  the  WTC  buildings.  This  maximized  the  precision  associated  with  contrasts  between  the  two 
building-specific  groups  of  occupants,  and  provided  a  nominal  level  of  precision  for  statistically 
analyzing  the  characteristics  of  survivors  from  each  building.  Robert  Santos,  NuStats,  developed  this 
sampling  approach  and  the  design  parameters  for  it. 

2.1.1         Sampling  Approach 

The  sampling  frame  (i.e.,  the  list  from  which  the  sample  was  drawn)  consisted  of  the  names  of  occupants 
from  badge  lists  of  WTC  1  and  WTC  2.  All  occupants  of  the  WTC  were  required  to  provide  personal 
data  in  support  of  issuing  badges  to  clear  through  the  security  station  at  the  entrance  of  each  tower.  The 
badge  lists  were  provided  to  NIST  by  the  Port  Authority  of  New  York  /New  Jersey.  For  sampling 
purposes,  the  lists  provided  name,  floor  of  occupancy,  employer,  and  social  security  number. 
Unfortunately,  contact  information  was  not  provided,  which  added  unanticipated  complexity  to  the  CATI 
survey  task.  In  addition,  the  badge  list  contains  September  1 1  occupants,  occupants  who  were  absent  on 
the  day  of  the  attacks,  decedents,  and  fonner  occupants. 

As  mentioned,  a  proportionate  stratified  sample  design  was  developed  for  this  survey  effort.  The  sample 
was  stratified  on  two  variables:  (1)  Building  Category  (WTC  1  vs.  WTC  2);  and  (2)  Floor  of  Occupancy 
(tliree  groupings).  This  resulted  in  six  pvimaiy  strata  as  defined  below: 

•  WTC  1  -  floors  0-43 

•  WTC  1  -  floors  44-76 


•  WTC  1  -  floors  77  and  above 

•  WTC  2  -  floors  0-43 

•  WTC  2  -  floors  44-76 

•  WTC  2  -  floors  77  and  above. 

Secondary  stratification  involved  two  categories  of  Tenant  Square  Footage  of  a  Floor:  (1)  occupants 
employed  by  tenants  occupying  up  to  40  percent  of  a  floor's  square  footage,  and  (2)  occupants  employed 
by  tenants  occupying  more  than  40 percent  of  a  floor's  square  footage. 

Given  that  the  sampling  frame  contained  both  eligible  and  ineligible  respondents,  an  intensive  screening 
effort  was  needed  to  identify  "eligible"  badge  list  members  -  namely,  those  who  were  inside  WTC  1  or 
WTC  2  during  the  attacks.  Moreover,  the  absence  of  telephone  numbers  for  the  badge  holders  on  the  list 
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necessitated  a  tracking/locating  effort.  The  primary  tracking  mechanism  was  to  search  public  databases 
using  the  LexisNexis  program."  This  necessitated  a  large  sample  to  generate  the  800  desired  interviews. 

The  number  of  occupant  selections  drawn  into  the  sample  was  contingent  on  four  key  design  parameters: 

•  the  percentage  of  individuals  from  badge  listings  for  whom  a  working  telephone  number  can 
be  found  (Initial  estimate:  80  percent  tracking  success) 

•  the  percentage  of  badge  listings  that  correspond  to  a  surviving  WTC  1 ,  WTC  2  occupant  on 
September  1 1  (Initial  estimate:  14  percent) 

•  the  cooperation  rate  for  screening  the  occupants  (Initial  estimate:  65  percent) 

•  the  interview  response  rate  among  9/1 1  survivors  (Initial  estimate:  50  percent). 

Given  these  design  parameters,  a  total  sample  of  approximately  22,735  persons  from  the  badge  list 
needed  to  be  tracked  and  located  to  generate  the  desire  800  completed  interviews.  A  reserve  sample  of 
20  percent  (or  about  n=4,550)  was  added  in  the  event  additional  sample  was  needed  due  to  unanticipated 
circumstance  (e.g.,  the  eligibility  rate  is  lower  than  anticipated).  The  reserve  was  held  "in  reserve"  while 
the  main  sample  was  worked.  Working  the  main  sample  allowed  preliminary  estimates  of  all  design 
parameters  to  be  monitored  so  that  an  infomied  decision  could  be  made  on  the  necessity  of  releasing 
none,  some  or  all  of  the  reserve.  Equal  samples  were  drawn  from  the  collections  of  badge  holders  in 
WTC  1  and  WTC  2.  That  is,  half  of  the  22,735  selections  were  allocated  to  the  WTC  1  sample,  and  the 
other  half  went  to  the  WTC  2  sample.  The  reserve  samples  were  similarly  drawn. 

2.1.2        Instrument  Development 

NuStats  developed  the  telephone  instrument  in  close  association  with  NIST  project  staff  and  the  expert 
panel  members.  A  one-day  meeting  of  these  individuals  was  held  at  NIST  in  July  2003  to  review  an 
instrument  draft.  Subsequent  to  this  meeting,  NuStats  prepared  a  conceptual  map  of  the  instmment  to 
prioritize  measurement  areas.  Subsequent  in-person  and  teleconference  meetings  relied  on  this 
conceptual  map  to  refine  the  measurement  areas  and  finalize  the  draft  instrument.  The  draft  instrument 
was  provided  to  Dr.  Jon  Krosnick,  a  cognitive  psychologist  and  nationally  recognized  expert  in 
questionnaire  design,  for  review  in  terms  of  controlling  for  non-response  and  measurement  errors  and  to 
Jamie  Abelson  for  review  for  human  subjects  and  surveyor  training  implications.^  After  their  review  and 
subsequent  final  reviews  by  NIST  project  staff  and  the  expert  panel,  the  CATI  instrument  was  finalized. 

The  interview  was  designed  to  measure  five  primary  content  areas,  presented  to  the  respondent  in  the 
following  order: 

•  Preparedness  and  Training  -  These  questions  collected  information  on  the  respondents' 
degree  of  evacuation  or  emergency  training  pertinent  to  their  September  experience.  The 


^  LexisNexis  is  a  web-based  search  product  that  enables  subscribers  to  search  nationwide  databases  to  locate  information  on 

individuals,  verify  addresses,  and  find  telephone  numbers. 
*  Dr.  Jon  Krosnick,  Ohio  State  University  Departments  of  Psychology  and  Political  Science;  Jamie  Abelson,  University  of 

Michigan  Institute  for  Social  Research. 
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questions  also  addressed  familiarity  with  the  WTC  buildings  based  on  length  of  employment 
at  the  location  and  also  whether  the  respondents  were  at  the  WTC  in  1993  during  that  prior 
bombing  incident. 

•  Initial  September  1 1  Experience  -  These  questions  captured  data  on  respondents'  initial 
awareness  that  something  had  happened  to  the  building  -  where  they  were,  what  they  were 
doing,  how  they  first  became  aware,  sources  of  information  about  the  event  (both  in  terms  of 
other  persons  and  also  in  terms  of  their  physical  environment). 

•  Interim  September  1 1  Experience  -  These  questions  addressed  the  span  between  the  time 
respondents  first  became  aware  of  the  WTC  incident  and  when  they  began  evacuating  -  data 
items  included  their  social  environment,  physical  enviroimient,  information  sources,  and 
factors  influencing  their  decision  to  evacuate. 

•  Evacuation  Experience  on  September  1 1  -  These  questions  captured  information  on  the 
respondents'  egress  from  the  building,  including  the  factors  that  facilitated  their  egress  and 
those  that  hindered  their  egress. 

•  Respondent  Characteristics  -  Infomiation  collected  included  age,  gender,  language,  and 
mobility  impairments. 

A  pilot  test  of  the  CATI  instrument  was  conducted  in  November  2003,  directed  by  Heather  Contrino, 
NuStats.  Eleven  interviews  were  completed.  In  the  pilot,  all  aspects  of  the  data  collection  plan  were 
evaluated,  including,  instrument  flow,  respondent  comprehension,  interviewer  administration, 
identification  of  questions  needing  probes  or  clarifications,  data  coding  and  processing,  and  review  of 
survey  data.  Only  very  minor  modifications  to  the  instrument  were  required  based  on  pilot  outcomes. 

2.1.3        CATI  Interviewing 

Highly  experienced  interviewers  at  DataSource  conducted  the  telephone  interviews  using  a  computer 
program  which  provided  questions  and  answer  categories.  Prior  to  calling,  subjects  received  an  advance 
letter  that  outlined  the  scope  and  purpose  of  the  investigation,  the  purpose  of  the  interview,  and  the 
telephone  call  that  came  several  days  later.  A  full  informed  consent  statement  appeared  in  the  letter,  as 
well.  When  interviewers  reached  the  respondents  by  telephone,  respondents  were  provided  a  description 
of  the  survey,  the  confidentiality  of  responses,  the  length  of  the  interview,  and  the  voluntary  nature  of 
participation.  Respondents  were  then  asked  if  they  wished  to  participate,  thereby  obtaining  oral  informed 
consent. 

Interviewer  training  was  held  on  December  1  and  2,  2003,  and  actual  dialing  of  the  sample  began  on 
December  3.  Interviewing  continued  until  January  23,  2004,  with  a  short  hiatus  during  the  December 
holidays.  The  actual  productivity  of  the  CATI  interviewing  was  lower  than  initial  design  parameters. 
Interviewing  productivity  rates  are  provided  in  Chapter  3  of  this  report.  In  December  2003,  pre- 
notification  letters  were  inadvertently  mailed  to  approximately  47  decedent  persons.  This  event  occurred 
because  the  badge  list  was  not  properly  flagged  for  decedents.  Once  the  error  surfaced,  the  database  of 
respondents  (i.e.,  the  badge  list)  was  manually  matched  and  checked  against  known  decedents  to  ensure 
that  all  potential  decedents  in  the  sample  were  flagged  and  removed  from  the  active  sample.  Letters  of 
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apology  were  mailed  to  decedent  households.  In  total,  803  CAT!  interviews  were  complelcd.  Details  on 
the  outcomes  of  the  CATI  interviewing  are  presented  in  Chapter  3. 

2.2     FACE-TO-FACE  INTERVIEWS 

The  objective  of  the  face-to-face  interviews  was  to  gather  first-hand  accounts  and  observations  of  the 
activities  and  events  inside  the  WTC  buildings  on  the  morning  of  September  1 1 .  The  qualitative  nature  of 
the  face-to-face  interviews  identified  heretofore  unknown  information,  evaluated  technical  hypotheses, 
and  explored  conscious  and  subconscious  motivations  for  occupant  and  responder  behaviors,  while 
allowing  for  qualitative  comparisons  to  the  telephone  interview  data. 

The  contract  required  that  the  face-to-face  interviews  be  conducted  via  a  relative  new  methodology,  that 
was  a  synthesis  of  the  Behavioral  Sequence  Interview  Technique  (BSIT)  originally  developed  by  Keating 
and  Loftus,'  and  the  Cognitive  Interviewing  Methods  (CIM),  originally  developed  by  Fisher^  and 
Gieselman.^  Both  approaches  begin  by  allowing  the  respondent  to  retell  an  unimpeded  account  without 
interruption  from  the  interviewer,  and  both  initially  employ  a  chronological  retelling  of  infonnation.  The 
BSIT  was  designed  to  yield  a  database  of  qualitative  infonnation  that  could  be  subjected  to  systematic 
analysis  and  consolidation,  while  CIM  was  designed  to  facilitate  investigative  interviews.  Since  the  WTC 
investigation  was  pursuing  both  goals  (i.e.,  creation  of  a  database  of  evacuation-related  behaviors  and  an 
investigatory  attempt  to  capture  information  relevant  to  outcomes),  the  face-to-face  methodology  was 
designed  to  combine  these  two  approaches. 

2.2.1         Sampling  Approach 

The  following  special  groups  of  building  occupants  were  targeted  in  the  face-to-face  interviews.  NIST 
and  the  expert  panel  selected  these  special  groups  for  the  face-to-face  interviews  because  these  groups 
were  beheved  to  hold  unique  pieces  of  information  to  add  to  the  investigation.  They  either  belong  to 
special  subpopulation  groups  or  were  in  locations  during  the  evacuation  that  enabled  them  to  experience 
distinctive  situations.  Because  these  respondents  were  expected  to  represent  relatively  rare  types  of 
occupants  or  other  targeted  groups,  a  snowball  sampling  approach  was  used  to  locate  respondents. 
Snowball  sampling  relies  on  referrals  from  initial  subjects  to  generate  additional  subjects.  In  the  case, 
referrals  came  from  NIST  public  announcements  (at  press  conferences  in  New  York  City  and  on  the  NIST 
web  site),  from  CATI  interview  respondents,  and  from  associated  parties  (i.e.,  NIST,  Port  Authority  of 
New  York/  New  Jersey).  The  selection  of  this  approach,  a  non-probabihty  design,  eliminated  the 
potential  for  generalizing  to  the  population  of  inference  (i.e.,  occupants  of  WTC  towers  on  September  1 1) 
but  the  design  provided  access  to  rare,  hard-to-fmd  persons  with  unique  information  to  provide  to  the 
investigation.  The  special  groups  targeted  in  the  face-to-face  interviews  are  described  as  follows. 


^  Keating,  F.P.;  Loftus,  E.  L.  Post  Fire  Interviews.  1984.  Development  and  Field  Validation  ofBeJiavioral  Sequence  Inten'iew 
Technique.  Final  Report.  NBS  GCR84-477.  Washington,  DC:  National  Bureau  of  Standards. 

^  Fisher,  R.P.,  Falkner,  K.L.,  Trevisan,  M.,  McCauley,  M.R.  2000.  Adapting  the  cognitive  interview  to  enhance  (35  years)  recall 
of  physical  activities.  Journal  of  Applied  Psychology,  85(2),  1 80-1 89. 

^  Gieselman,  R.E.,  Fisher,  R.,  Mackinnon,  D.,  Hallond,  H.  1986.  Enhancement  of  eyewitness  memoiy  with  the  cognitive 
interview.  American  Journal  of  Psychology,  99,  385-401 . 
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Occupants  Subgroup  A:  People  near  the  Floors  of  Impact 

•  50  people  from  floors  90  -  94  in  WTC  1  at  the  time  of  WTC  1  impact. 

•  50  people  from  floors  75  -  110  in  WTC  2  at  the  time  of  WTC  2  impact. 

Rationale  for  inclusion:  These  people  witnessed  fire,  smoke,  structural/  system  damage,  heard 
PA  aimouncements,  and  walked  the  greatest  distance  to  evacuate  the  buildings. 

Occupants  Subgroup  B:  People  in  the  Lobby  who  Witnessed  Fireballs 

•  10  people  who  witnessed  fireballs. 

Rationale  for  inclusion:  Estimating  the  size  and  source  of  the  fireball  to  identify  the  path  the  fuel 
traveled,  how  much  fuel  was  consumed,  and  how  much  damage  to  the  lobby  resulted. 

Occupants  Subgroup  C:  Families  of  Victims  who  Talked  to  Victim  during  Disaster. 

•  30  family  members  or  others. 

Rationale  for  inclusion:  These  people  who  communicated  with  victims  may  be  the  sole  source  of 
"first-hand"  information  from  above  the  floors  of  impact. 

Occupants  Subgroup  D:  People  in  WTC  Building  7 

•  1 5  people  who  evacuated 

•  5  people  with  some  level  of  decision-making  authority  over  the  evacuation. 

Rationale  for  inclusion:  Determine  the  total  evacuation  time  for  Building  7,  total  number  of 
people  evacuated,  density  of  people  in  the  stairwells,  and  evaluate  when  fires  may  have  started. 

Occupants  Subgroup  E:  Persons  with  Building  Responsibility 

•  50  floor  wardens,  fire  safety  directors,  or  building  management  staff. 

Rationale  for  inclusion:  Involved  in  decision-making  process  on  September  1 1  about  who  and 
how  to  evacuate  the  buildings.  Know  who  had  what  information,  when  they  had  it,  and  what  they 
did  with  it. 

Occupants  Subgroup  F:  People  w  ho  Used  or  were  Trapped  in  Elevators 

•  25  people  trapped  in  elevators  or  used  elevators  after  their  building  was  hit. 

Rationale  for  inclusion:  Estimating  the  number  of  people  who  may  have  been  killed  inside 
elevators  and  what  the  failure  mechanism  of  the  elevator  system  was. 
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Occupants  Subgroup  G:  People  with  Disabilities 

•  20  people  who  experienced  a  mobility  impairment  prior  to,  or  during,  their  evacuation  - 
anything  from  a  disability  under  the  Americans  with  Disabilities  Act  (ADA)  to  pregnancy, 
overweight,  age,  etc. 

Rationale  for  inclusion:  Document  the  challenges  faced  by  people  with  disabilities  in  order  to 
better  address  those  needs  in  future  building  incidents  as  well  as  identify  pro-social  behaviors, 
rescue,  or  devices  that  contributed  to  their  ultimate  evacuation  success. 

Occupants  Subgroup  H:  Random  Evacuee 

•  25  people  from  each  of  the  lower  and  middle  thirds  of  WTC  1 

•  25  people  from  each  of  the  lower  and  middle  thirds  of  WTC  2  (that  is,  not  from  the  floors 
near  the  impact).  4  zones  x  25  people  =  100  people. 

Rationale  for  inclusion:  Document  typical  evacuation  experience.  Ensure  unanticipated  events 
or  experiences  were  captured. 

2.2.2        Instrument  Development 

NuStats  developed  the  face-to-face  instrument  in  close  association  with  NIST  project  staff  and  the  NIST 
expert  panel  members.  However,  after  this  draft  was  prepared  it  became  apparent  that  the  prescribed  data 
collection  protocol  for  the  face-to-face  interviews  would  determine  the  instrument  structure  and  content. 
Thus,  the  instrument  was  drafted  as  a  computer-assisted  personal  interview  (CAPI)  application  that  would 
operate  on  Pen  Tablet  PCs  rather  than  as  a  paper  document  (see  Appendix  A,  Section  A. 3. 2.1).  With 
CAPI  technology,  high  quality  qualitative  data  were  electronically  coded,  organized,  and  stored  directly 
in  an  electronic  format,  with  all  relevant  error  checking  performed  automatically.  Not  only  was  this 
application  more  appropriate  for  the  prescribed  data  collection  protocol,  this  process  simultaneously 
ehminated  unnecessary  data  entry  costs  and  associated  key  entry  errors  that  typically  occur  in  paper-based 
field  surveys.  Both  the  Pen  Tablet  PCs  and  the  application  were  provided  by  GeoStats. 

The  CAPI  application  captured  information  from  participants  in  four  (4)  steps. 

Step  1:  Uninterrupted  Narrative  Account 

The  participants  were  asked  to  relate  their  experience  on  September  1 1  from  the  time  that  they  first  knew 
something  was  wrong  to  the  time  when  they  exited  the  building.  Researchers  and  practitioners  involved 
with  cognitive  interviewing  believe  that  starting  the  face-to-face  interviews  in  this  manner  both  improves 
recall  and  helps  build  rapport  between  the  participant  and  the  interviewer.  The  surveyors  were  trained  to 
listen  intently  and  to  take  notes  during  this  stage. 

Step  2:  Collaborative,  Structured  Action-Cue-Goal  Account 

After  the  participant  completed  their  story,  the  respondent  was  asked  to  re-visit  their  experience. 
Interviewers  worked  cooperatively  with  participants  to  record  entries  into  a  CAPI  table  (see  Table  2-1). 
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During  this  collaboration,  interviewers  recorded  the  infonnation,  guided  the  process,  and  asked  questions 
to  clarify  and  define  the  content.  In  the  structured  table,  each  row  of  the  table  was  a  triad  comprised  of  an 
action,  a  cue,  and  a  goal. 


Table  2-1.  Example  tabular  face-to-face  interview  data  entry. 


Action 

Cue 

Goal 

1  stumbled  in  the  dark  towards 
where  I  thought  the  voice  came 

I  heard  someone  yell  "I've  found 
a  clear  path" 

So  that  I  could  find  a  way  to 
escape 

So  1  called  out  to  whoever  yelled, 
"I'm  near  the  reception  area. 
Where  are  you?" 

My  path  was  blocked  by  debris 

To  try  to  get  a  better  idea  of 
where  the  person  was 

•  Action.  An  action  was  something  that  the  participant  did  as  a  reaction  to  a  cue.  Actions 
should  be  specific  enough  for  a  reader  to  visualize. 

•  Cue.  A  cue  was  something  that  starts  an  action.  Cues  can  be  external  (something  one  sees, 
hears,  smells,  feels)  or  something  internal  (a  decision,  a  memory  ). 


•    Goal.  Goals  were  what  the  participant  was  trying  to  accomplish  by  taking  the  action. 
Step  3:  Review  and  Edit  Account  of  September  11  Experience 

After  the  table  was  completed  collaboratively,  the  interviewer  asked  the  participant  to  review  what  had 
been  recorded  to  ensure  that  it  accurately  captured  the  participant's  experience. 

Step  4:  Topic  Cards 

After  completing  the  negotiated,  structured  narrative  account,  the  interviewer  showed  the  participant 
"Topics"  cards  and  asked  the  participant  to  identify  those  topics  that  were  relevant  to  his/her  experience. 
The  topics  were  intended  to  capture  specific  information  of  particular  value  to  the  investigation.  If  a  topic 
was  relevant,  the  interviewer  recorded  the  open-ended  statements  of  the  participants  in  text  boxes  that 
were  associated  with  the  relevant  A-C-G  row  in  the  structured  table.  The  interviewing  application  was 
programmed  to  enable  this  to  be  easily  done. 

On  September  9  and  10,  2003  in  New  York  City,  NuStats  interviewed  three  WTC  building  occupants 
with  the  purpose  of  beta  testing  the  face-to-face  CAPI  application.  These  beta  test  interviews  were 
conducted  by  Dr.  Johanna  Zmud  and  Ms.  Delia  Santos.  Results  of  the  beta  test  were  compiled  and 
delivered  to  NIST.  The  three  individuals  were  paid  as  consultants  to  NuStats  because  IRB  approval  had 
not  been  obtained  at  the  time.  The  interviews  ranged  from  1  to  3  hours.  Respondents  reacted  quite 
positively  to  the  "cue",  "action",  and  "reason"  (C-A-R)  protocol.  The  act  of  being  interviewed  on  their 
September  1 1  experiences  was  an  emotionally  trying  activity  for  the  respondents.  Respondents  arrived  at 
the  interview  site  emotionally  stressed.  Their  unimpeded  narratives  were  spoken  with  emotion  and 
sometimes  their  eyes  teared.  The  C-A-R  protocol,  however,  seemed  to  objectify  the  experience  for  them. 
Thus,  the  negotiated  description  of  their  experience  was  done  with  much  less  emotion  than  the  unimpeded 
narrative  and  greater  detail  resulted.  Generally,  the  unimpeded  narratives  took  approximately  one-quarter 
the  time  of  the  negotiated  C-A-R  activity.  The  respondents  appeared  genuinely  to  be  engaged  in  working 
with  the  interviewer  to  complete  the  C-A-R  table.  Respondents  also  took  ownership  of  the  content  of  the 
C-A-R  table.  The  beta  test  identified  some  mechanical  problems  with  the  application  and  also  some 


10 


NISTNCSTAR  1-7B,  WTC  Investigation 


Summary  of  Data  Collection  Outcomes 


conceptual  issues  pertaining  to  clarifying  C-A-R.  Revisions  to  the  application  and  to  the  conceptual 
underpinnings  of  the  CAP!  application  were  made. 

A  pilot  test  of  the  revised  CAPI  application  was  conducted  on  November  5  and  6  in  NYC.  NuStats 
NY-based  coordinator  (Ms.  Maria  Elena  Ramos)  recruited  five  respondent  consultants  for  these  pilot 
interviews.  The  interviews  were  conducted  by  Dr.  Johanna  Zmud  and  Delia  Santos.  The  pilot  revealed 
that  the  modifications  to  the  application  enhanced  its  user-interface. 

2.2.3        Face-to-Face  Interviews  with  Occupants 

The  face-to-face  interviews  were  conducted  by  surveyors  recruited  by  a  subcontractor,  Diversity  Services 
in  New  York  City.  Surveyors  were  sought  who  had  some  social  work,  interviewing,  or  client-interaction 
experience.  Training  of  the  surveyors  was  held  over  a  3-day  period  in  early  November  2003  in  New  York 
City.  Surveyors  were  required  to  attend  all  three  days  of  training.  Surveyors  were  responsible  for  setting 
up  the  interview  appointments,  completing  the  interviews,  and  uploading  interview  files  to  the  secure 
NuStats  file  transfer  protocol  (FTP)  site  for  data  editing  and  processing. 

Through  Diversity  Services,  NuStats  trained  12  surveyors  of  which  ten  continued  interviewing  through 
mid-  February  2004.  A  skeleton  crew  of  five  surveyors  continued  interviewing  through  the  end  of 
April  2004  for  completing  face-to-face  interviews  with  really  hard-to-find  special  groups,  to  recruit  focus 
group  participants,  and  to  conduct  coding  of  9-1-1  tapes  and  transcripts  of  interviews  with  first  responders 
as  a  special  assignment  from  NIST. 

Respondents  for  the  face-to-face  interviews  were  identified  through  three  avenues:  (1)  NIST,  (2)  Hotline, 
(3)  CATI  survey.  From  these  sources,  persons  were  identified  with  whom  a  DataSource  interviewer 
would  conduct  a  telephone  screener  interview.  Respondents  who  successfully  completed  a  screener 
interview  were  coded  based  on  their  affiliation  with  one  or  more  of  the  target  groups  and  placed  into  the 
sampling  frame.  The  day-to-day  management  of  this  sample  was  critical  to  completing  interviews  with 
this  hard-to-reach  population.  The  sample  file  was  housed  on  the  NuStats'  FTP  site.  The  NYC-based 
coordinator  downloaded  sample  information  daily  and  distributed  the  new  sample  to  surveyors. 
Surveyors  scheduled  interviews  and  emailed  the  information  back  to  the  coordinator,  who  then  posted  it 
on  the  NuStats'  FTP  site  so  that  the  information  on  potential  and  completed  interviews  was  kept  in  real- 
time. The  information  on  completed,  missed,  or  rescheduled  interviews  was  tracked  at  NuStats  and  a 
report  was  prepared  three-times  per  week  for  NuStats  and  NIST  project  staff. 

Face-to-face  interviewing  began  in  mid-November.  Each  surveyor  was  monitored  in  a  "live"  interview 
prior  to  being  allowed  to  operate  independently.  A  group  debrief  of  surveyors  was  held  in  late  November 
to  trouble-shoot  any  challenges  that  had  arisen  in  the  first  few  interviews  and  also  to  ensure  that  all 
surveyors  understood  the  interviewing  requirements  and  were  successfully  carrying  these  out.  The  face- 
to-face  interviews  began  slowly  due  to  lack  of  sample.  Most  of  the  referrals  for  these  interviews  were  to 
come  from  the  CATI  survey  and  that  did  not  start  until  early  December  2003.  During  this  interim  period, 
interviews  were  conducted  with  the  trickle  of  respondents  who  entered  the  frame  through  the  Hotline. 
Interviewing  continued  through  the  end  of  April  to  maximize  the  ability  to  locate  and  find  potential  face- 
to-face  interview  respondents. 
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In  all,  NuStats  had  access  to  744  potential  face-to-face  interview  respondents,  the  majority  of  these 
coming  from  the  screening  of  respondents  in  the  CATI  survey.  The  face-to-face  sample  pool  included: 

•  456  cases  from  the  CATI,'" 

•  140  cases  from  the  published  accounts  database," 

•  106  cases  from  the  Port  Authority's  badge  list  for  WTC  7,  and 

•  42  referrals  from  a  variety  of  sources  including  NIST. 

All  of  these  cases,  other  than  the  cases  from  the  CATI,  required  intensive  tracking  using  LexisNexis  to 
locate  contact  infonnation.  Approximately  70  percent  of  persons  in  the  sample  pool  were  located  and 
contacted.  One  of  the  anticipated  challenges  in  this  interviewing  effort  was  the  "no-show"  rate  for 
interview  appointments,  given  the  sensitive  nature  of  the  interviews.  During  December  and  January, 
NuStats  experienced  a  no-show  rate  of  1  in  4  (25  percent).  However,  starting  in  February  the  no-show 
rate  continued  to  increase  to  more  than  1  in  3  (36  percent).  This  was  attributed  to  a  shrinking  pool  of 
available  participants;  most  of  them  being  persons  who  previously  cancelled  or  were  no-shows.  As 
people  were  successfully  interviewed,  the  more  reluctant  subjects  were  the  ones  who  remained  to  be 
interviewed;  thus,  increasing  the  likelihood  of  a  "no-show"  appointment.  In  March  and  April, 
interviewers  were  trying  to  interview  the  "hardest-to-get"  cases. 

Interviewing  was  done  at  various  locations,  depending  on  the  preference  of  the  respondent.  The  goal  was 
to  maximize  participation  in  the  face-to-face  interviews  by  making  the  interviewing  situation  as 
comfortable  and  convenient  as  possible  for  the  potential  respondents.  Interviewing  locations  included  the 
Diversity  Services  offices  in  midtown  Manhattan,  NuStats'  procured  space  in  downtown  Manhattan,  as 
well  as  the  work  or  residential  locations  of  respondents.  About  20  percent  of  interviews  were  conducted 
at  off-site  locations.  Surveyors  were  given  personal  Pen  Tablet  PC's  to  accommodate  interviewing  at 
these  multiple  locations.  The  average  length  of  the  face-to-face  interviews  was  3  hours. 

2.2.4        Face-to-Face  Interviews  with  Family  Members 

The  conduct  of  the  face-to-face  interviews  with  family  members  was  particularly  delicate  to  implement 
because  of  the  emotional  duress  that  many  of  the  individuals  still  felt.  The  names  and  contact  information 
(for  some  not  all)  of  16  potential  respondents  were  provided  to  NuStats  by  NIST.  NuStats  mailed  pre- 
notification  letters  to  those  cases  where  the  LexisNexis  search  provided  address  of  known  names  of 
family  members  of  victims.  These  cases  were  followed  up  by  a  telephone  call  to  the  person.  In  all  but 
one  case,  the  8  completed  interviews  with  the  family  members  of  victims  were  conducted  by  telephone  by 
Delia  Santos,  NuStats.  One  interview  was  conducted  in-person  by  Dr.  Johanna  Zmud,  NuStats. 


These  were  persons  who  met  the  screening  criteria  and  agreed  to  be  interviewed  again  using  the  face-to-face  protocol. 
"  The  published  accounts  database  was  developed  by  Dr.  Rita  Fahy  and  Dr.  Guylene  Proulx,  and  published  as  NIST 
NCSTAR  1-7A. 
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2.3     FOCUS  GROUP  INTERVIEWS 

The  goal  of  the  focus  groups  was  to  elicit  group  representations  of  specific  events  or  themes.  Distinct 
categories  of  persons  were  provided  by  NIST  based  on  preliminary  analyses  of  CATI  survey  and  face-to- 
face  interview  data.  These  groups  were  prioritized  based  on  expectations  of  sample  availability  and 
information  value  as  presented  below.  The  goal  was  to  recruit  five  to  six  persons  for  each  group. 

Priority  Group  1 

1.  Mobihty  Challenged  -  WTC  1  only.  At  least  half  of  the  participants  were  to  be  wheelchair  bound 
or  use  some  type  of  device  to  aid  their  mobility.  Half  could  have  less  visible  mobility  challenges, 
such  as  asthma.  Questioning  priorities  were  to  (1)  have  them  tell  their  stories,  (2)  talk  about  the 
building  design  or  evacuation  procedure  policies  that  aided  their  evacuation  or  (3)  hindered  their 
evacuation,  and  (4)  what  would  they  like  to  see  in  terms  of  building  design  or  evacuation 
procedures  policies  in  the  future.  These  persons  would  be  recruited  from  the  face-to-face 
interview  sample. 

2.  Persons  near  the  floor  of  impact  in  WTC  1  (floors  88  -  94).  This  group  provided  explanatory 
power  to  the  causal  models  on  evacuation  behavior.  Dependent  variables  in  the  models  are: 

(1)  length  of  time  between  their  decision  to  evacuate  and  their  actual  evacuation  time,  and 

(2)  time  out  of  the  building  once  they  started  their  evacuation.  NIST  provided  variables  that  were 
identified  as  independent  variables  in  the  model  so  that  these  could  be  probed  during  the  group 
discussion.  These  persons  would  be  recruited  from  the  face-to-face  interview  sample. 

3.  WTC  1  respondents  who  started  evacuating  2  standard  deviations  after  the  mean  evacuation  time. 
The  "glue"  of  this  group  was  to  examine  why  some  people  took  so  long  to  start  evacuating. 
These  persons  would  be  recruited  from  the  CATI  sample. 

Priority  Group  2 

1.  12  Persons  from  floor  78  (above  impact)  in  WTC  2  who  survived  the  impact.  It  would  be 
important  to  interview  these  persons  as  face-to-face  interviews  first  to  know  what  they  did  to 
survive.  A  Lexis  Nexus  search  was  performed  in  order  to  locate  these  individuals. 

2.  Group  who  stayed  behind  in  WTC  I,  floor  68  for  20-45  minutes  after  impact.  Information 
priority  was  to  find  out  why  they  stayed  behind  and  did  not  start  evacuating  right  away.  These 
persons  would  be  recruited  from  CATI  or  face-to-face  completed  interviews. 

3.  Floor  wardens  in  either  WTC  1  or  WTC  2.  These  were  persons  who  "tried"  to  perform  their 
warden  duties.  Information  priorities  were  to  probe  on  why  people  might  have  ignored  them, 
what  they  felt  might  have  been  a  better  evacuation  procedures  than  the  one  that  they  employed,  if 
they  thought  the  evacuation  procedures  /  policies  they  followed  were  sound  and  should  be 
guidance  for  future  events.  These  persons  would  be  recruited  from  CATI  or  face-to-face 
completed  interviews. 

Priority  Group  3 

1 .  People  redirected  in  stairs  in  WTC  I .  These  were  persons  who  were  evacuating  down  the  stairs 
and  were  redirected  to  another  stairwell.  The  information  value  would  be  in  knowing  who  did  the 
redirecting  and  why.  These  persons  would  be  recruited  from  face-to-face  interview  sample. 

2.  People  who  were  evacuating  down  the  stairwells  and  were  stopped  on  the  sky  lobby  on  floor  78 
in  WTC  2  before  the  plane  hit  and  told  to  return  to  their  floors.  Questions  would  focus  on  who 
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did  the  redirecting;  why;  and  people's  reactions  to  being  redirected.  These  persons  would  be 
recruited  from  the  face-to-face  interviews. 

3.  Building  support  staff.  These  were  elevator  operators,  maintenance  persons,  etc.,  to  get  their 
assessment  of  the  building  design  features  that  aided  or  hindered  successful  evacuation.  They 
would  be  recruited  through  snowball  sampling,  outside  of  the  CATI  or  face-to-face  interview 
sample. 

4.  Evacuation  helpers.  These  were  persons  in  either  WTC  1  or  2  who  helped  an  injured  person,  a 
mobility  challenged  person,  or  anyone  else  during  their  evacuation  to  get  the  helper  perspective 
on  what  building  design  or  evacuation  procedure  policies  helped  or  hindered  their  efforts.  They 
would  be  recruited  from  face-to-face  interview  sample. 

5.  Persons  who  observed  the  fireball  in  lobby  of  WTC  1 .  This  was  a  low  priority  because  of  the 
lack  of  sample  to  constitute  a  group. 

2.3.1         Focus  Group  Recruitment  and  Interview  Protocol 

After  prioritizing  and  analyzing  the  availability  of  sample  with  which  to  recruit  focus  group  participants, 
it  was  decided  to  conduct  six  focus  groups:  (1 )  WTC  1  occupants  on  March  9,  2004,  (2)  WTC  2 
occupants  on  April  1,  2004,  (3)  mobility  challenged  persons  on  April  3,  2004,  (4)  persons  with  building 
responsibility  on  April  27,  2004,  (5)  persons  near  the  floors  of  impact  on  April  28,  2004,  and  (6)  persons 
with  floor  warden  responsibilities  on  May  12,  2004.  Even  though  these  groups  were  vetted  for  inclusion 
based  on  the  availability  of  sample,  it  was  still  a  challenge  to  locate  persons  who  were  willing  and  able  to 
attend.  The  original  plan  was  to  conduct  all  the  groups  during  one  week  in  New  York  City.  But  due  to 
the  challenge  in  recruiting  participants,  the  groups  were  spaced  out  as  noted  above.  In  most  cases, 
participants  had  already  completed  either  a  CATI  interview  or  a  face-to-face  interview.  Recruitment  of 
focus  group  participants  was  conducted  by  the  face-to-face  interviewers;  whenever  possible,  the 
recruitment  contact  was  made  by  the  surveyor  who  had  conducted  the  face-to-face  interview  with  the 
respondent. 

The  interview  protocols  for  the  focus  groups  followed  the  same  basic  structure,  but  each  was  customized 
for  specific  infonnation  priorities.  Dr.  Carlos  Arce,  NuStats,  moderated  the  first  group.  The  remaining 
five  groups  were  moderated  by  Dr.  Johanna  Zmud,  NuStats.  Because  audio  taping  or  videotaping  was  not 
allowed,  two  note  takers  were  present  at  each  group  to  record  the  discussion.  Each  focus  group  lasted 
approximately  three  hours,  and  participants  were  reimbursed  for  their  travel  with  a  stipend  of  $100. 
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Interviews  were  completed  with  surveyors  of  the  WTC  attacks  via  CATI,  CAPI,  and  focus  groups.  The 
information  resulting  from  these  interviews  provided  unique  first-hand  accounts  of  experiences  in  the 
towers  on  the  morning  of  September  1 1,  2001.  The  level  of  effort  required  to  complete  interviews  with 
survivors  of  the  WTC  attacks  was  greater  than  anticipated.  There  were  several  factors  that  influenced 
NuStats'  ability  to  locate,  contact,  and  complete  interviews,  including  availability  of  sample  and  the 
willingness  of  survivors  to  revisit  their  experiences  in  an  interview  format.  The  sections  below  document 
outcomes  of  the  interviewing  efforts  and  the  factors  that  influenced  these  outcomes. 


3.1      TELEPHONE  INTERVIEWS 


A  total  sample  of  26,000  was  drawn,  comprising  13,000  names  for  each  tower.  From  this  sample,  CATI 
interviews  were  completed  with  803  persons.  Of  which,  427  persons  were  in  WTC  1  on  September  11, 
and  376  persons  were  in  WTC  2.  Using  guidehnes  of  the  Council  of  American  Survey  Research 
Organizations  (CASRO),  the  estimated  response  rate  was  23  percent.  This  response  rate  was  driven  by  an 
inability  to  contact  and  screen  eligible  respondents,  rather  than  by  refusals.  Of  those  contacted,  only 
7  percent  refused  to  be  interviewed.  On  the  other  hand,  only  47  percent  of  sampled  persons  were 
contacted,  of  which  only  1 8  percent  were  eligible  to  be  interviewed  (that  is,  they  were  in  the  towers  on  the 
morning  of  September  11,  2001). 

Table  3-1  summarizes  the  final  disposition  of  the  CATI  sample  and  the  total  (locating)  sample.  The  table 
is  comprised  of  two  sets  of  rows.  The  top  set  pertains  to  the  CATI  sample  and  represents  those  sample 
persons  for  whom  an  initial  telephone  number  was  identified  prior  to  commencing  the  CATI  survey 
operations.  The  bottom  set  of  rows  with  the  heading  "Total  Sample  Disposition"  represents  the  results  of 
locating/tracking  efforts  used  to  identify  usable  telephone  numbers  associated  with  the  sample  subjects. 
(Recall  that  only  name,  SSN,  and  employer  were  available;  no  other  contact  information  was  readily 
available.) 

The  top  set  of  rows  in  Table  3-1  presents  the  final  disposition  of  the  sample  by  tower  as  well  as  for  the 
overall  sample.  Several  statistics  in  the  percentage  distribution  (rightmost)  column  are  notable.  First, 
subjects  for  half  the  sample  (50.5  percent)  could  not  be  contacted,  due  either  to  failures  to  answer  the 
phone,  answering  machines,  unusable  numbers  (e.g.,  wrong  number,  disconnected,  business),  etc.  Most 
of  these  telephone  numbers  represent  "unloadable"  subjects  -  subjects  for  whom  the  initial  telephone 
number  was  incorrect.  It  bears  reiterating  that  substantial  additional  research  during  CATI  operations  was 
conducted  using  powerful  subscription-based  web-based  search  engines.  Unfortunately,  little  information 
was  available  for  these  individuals. 
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'able  3-1.  Disposition  of  the  CATI  samp 

e  and  the  total  samp 
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10,063 
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CATI  TOTAL 
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Total  Sample  Disposition 

WTC  1 

WTC  2 

Total 
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Found  initial  telephone  number 

9,986 

9,937 

19,923 

76.6  % 

Unable  to  find  a  telephone 
number 

3,014 

3,063 

6,077 

23.4  % 

SAMPLE  TOTAL 

13,000 

13,000 

26,000 
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a.  Table  data  is  weighted.  Tower  location  as  indicated  in  the  badge  list  and  may  differ  from  reported 
tower  location. 


The  bottom  set  of  rows  shows  that  telephone  numbers  were  identified  for  just  over  three  quarters 
(76.6  percent)  of  the  sainpled  subjects.  Moreover,  this  rate  was  fairly  uniform  across  towers.  The 
19,923  individuals  with  an  initial  telephone  number  were  then  loaded  into  the  CATI  sample  management 
system  for  calling.  Ultimately,  all  reserve  samples  were  used  in  the  telephone  survey.  In  the  initial 
design  parameters,  it  was  assumed  that  82  percent  of  the  subjects  would  be  locatable.  While  76.7  percent 
is  close,  many  of  the  numbers  were  obsolete  (e.g.,  disconnect,  wrong  number)  and  necessitated  additional 
tracking  during  CATI  operations.  UUimately,  by  the  end  of  data  collection,  only  half  the  sample 
represented  confinned  contacts  with  subjects. 

A  second  result  of  interest  is  the  prevalence  of  ineligible  subjects — those  not  in  the  building  on  the 
morning  of  September  1 1,  2001.  A  third  result  is  the  existence  of  decedents — some  from  the 
September  1 1  attack  and  others  from  causes  not  necessarily  related  to  September  1 1  (e.g.,  cause 
unknown,  natural  causes).  Most  of  the  September  1 1  decedents  were  encountered  due  to  a  difference  in 
the  full  (formal)  name  of  the  subject  and  the  name  that  appeared  on  the  badge  list  (e.g.,  the  badge  list 
sometimes  contained  maiden  names,  middle  names,  nicknames,  misspelled  first  or  last  names,  out  of 
sequence  names,  titles,  and  so  on).  This  impeded  the  ability  to  remove  known  decedents  prior  to  calling. 

The  outcome  of  CATI  operations  on  the  final  outcome  rates  is  presented  by  tower  in  Table  3-2.  The  table 
shows  screening  rates,  interview  rates,  and  rates  of  eligible  occupants  (among  those  who  responded  to  the 
screening  questions).  The  first  row  shows  that  screening  response  rates  were  relatively  uniform  across 
towers  at  about  46  percent.  A  screening  response  rate  of  65  percent  had  been  planned.  Similarly, 
interview  response  rates  (among  screened  eligible  subjects)  were  relatively  stable  across  towers  at  about 
49  percent.  This  is  consistent  with  the  planned  interview  response  rate  of  50  percent. 
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The  eligibility  rates  were  higher  than  expected — about  18  percent  overall  compared  to  the  14  percent 
expected.  The  eligibility  rate  among  WTC  1  subjects  was  slightly  higher  than  those  of  WTC  2. 
However,  the  overall  response  rates  are  essentially  uniform  across  towers,  at  22.6  percent. 


Table  3-2.  Summary  disposition  rates 

3y  tower. 

Disposition  Rate" 

WTC  1 

WTC  2 

Total 

Screen 

46.5  % 

45.8  % 

46.1  % 

Interview 

48.6  % 

49,5  % 

49.0  % 

Eligibility 

18.9% 

16.7% 

17.8  % 

Overall 

22.6  % 

22.7  % 

22.6  % 

a.  Definitions  for  "Rates"  consistent  with  American  Association  of  Public 
Opinion  Research  (AAPOR)  Standards,  which  may  be  found  at 
http://www.aapor.org/pdfs/standarddefs2004.pdf 


3.2     FACE-TO-FACE  INTERVIEWS 

The  face-to-face  interviews  were  conducted  via  non-probability  methods.  Thus,  response  rate  estimates 
cannot  be  derived.  In  total,  220  interviews  were  completed  between  mid-November  and  mid- April.  The 
following  is  the  breakdown  of  completes  by  group  affiliation:'" 


28 

=  Near  floors  of  impact 

2 

=  Persons  in  the  lobby  who  saw  fireballs 

8 

=  Family  of  victim 

7 

=  Persons  in  building  7 

33 

=  Persons  with  building  responsibility 

15 

=  Persons  in  elevators 

13 

=  Persons  with  disabilities 

•     114  =  Persons  who  evacuated  from  WTC  1  or  2  not  otherwise  coded. 

Most  of  the  interviews  were  conducted  with  persons  who  had  also  agreed  to  complete  a  CATI  interview. 
The  provision  of  sample,  outside  of  the  CATI  interviewees,  was  less  than  originally  anticipated. 
Snowball  sampling  proved  less  than  effective  as  survivors  were  reluctant  to  provide  names  of  other 
persons  who  may  have  qualified  for  a  face-to-face  interview  out  of  respect  for  the  persons'  privacy  or 
mental  health.  The  biggest  challenge  during  the  face-to-face  interviews  was  the  "no  show"  rate  for 
scheduled  appointments,  which  ranged  from  25  percent  at  the  start  of  interviewing  to  36  percent  as 
interviewing  ended.  Interviewers  averaged  10  interviews  per  week  in  December  and  January.  But  this 
productivity  dwindled  to  less  than  three  per  week  in  February,  March,  and  April  as  available  sample  and 


Some  individuals  may  have  had  more  than  one  group  affiliation,  but  each  individual  was  coded  to  a  primary  group  for  tracking 
purposes.  A  hierarchy  was  used  to  do  the  primary  coding  based  wherein  an  individual  was  coded  to  the  hardest-to-locate 
group. 
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willing  respondents  diminished.  Interviewers  also  found  that  they  were  required  to  travel  to  residences  or 
work  places  outside  of  Manhattan  for  these  harder-to-interview  persons. 


3.3     FOCUS  GROUP  INTERVIEWS 


Six  focus  groups  were  completed.  Planning  and  recruiting  for  all  focus  group  participants  began  in  mid- 
March.  Recruiting  for  some  focus  groups  was  more  challenging  than  others.  The  easiest  to  recruit  groups 
were  with  survivors  from  WTC  1  and  WTC  2  on  their  evacuation  experiences.  These  groups  were  held 
on  March  9,  2004  and  April  1,  2004.  Six  individuals  were  recruited  for  the  group  of  WTC  1  survivors, 
and  all  six  showed  at  the  facility.  Eight  persons  were  recruited  for  the  group  of  WTC  2  survivors  and 
seven  of  the  eight  showed. 

For  the  remaining  focus  groups,  it  was  challenging  to  track,  locate,  and  recruit  individuals.  The  focus 
group  with  mobility  challenged  individuals  was  held  on  April  3,  2004.  Five  individuals  were  recruited 
(of  which  two  were  wheelchair-bound),  and  three  individuals  showed  up,  of  which  one  was  wheelchair- 
bound.  Focus  groups  with  persons  having  building  responsibility  and  with  persons  near  the  floors  of 
impact  were  held  on  consecutive  days,  April  27  and  28,  2004.  Eight  individuals  were  recruited  for  the 
focus  group  of  persons  with  building  responsibility,  and  six  individuals  participated.  Six  persons  were 
recruited  for  the  group  of  persons  near  the  floors  of  impact,  and  four  participated.  Finally,  the  floor 
wardens  focus  group  was  the  most  difficult  to  recruit.  Only  four  persons  were  recruited,  and  two 
showed  up. 

The  venues  for  the  focus  groups  were  changed  from  Manhattan  to  outside  of  Manhattan  depending  on  the 
preferences  of  focus  group  respondents.  Among  respondents  who  were  willing  to  participate  in  the  focus 
group,  several  no  longer  traveled  into  Manhattan.  Of  the  persons  who  participated  in  the  groups,  all  were 
active  in  the  discussion  and  willing  to  share  their  experiences,  thoughts,  and  recommendations. 
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Data  File  Deliveries 


NuStats  was  responsible  for  delivery  of  databases  containing  information  resulting  from  the  interviews 
with  survivors.  These  databases  comprised  various  formats,  specific  to  the  type  of  interview  conducted, 
In  addition  to  delivering  databases,  NuStats  also  provided  database  support  services.  These  services 
included  locating  a  subcontractor  to  provide  qualitative  analysis  support  for  the  face-to-face  interviews 
and  providing  statistical  expertise  to  support  model  development  using  the  CATl  data. 


4.1      TELEPHONE  SURVEY  INTERVIEWS 

The  CATI  data  was  delivered  in  SPSS  and  DBF  formats  in  late  January  2004,  after  completing  processing 
and  quality  review.  After  delivery  of  the  final  data,  NuStats  participated  in  a  weeklong  analysis  session 
with  NIST  to  finalize  the  statistical  analysis  procedures.  During  this  time  and  subsequently,  NuStats 
provided  data  support  to  NIST  for  reporting  writing  and  final  data  review.  NuStats  also  provided 
statistical  analysis  support  during  the  months  of  February  and  March  for  the  development  of  a  dataset  that 
was  consistent  with  the  causal  modeling  requirements  and  the  running  of  necessary  statistical  programs  to 
support  quantitative  analysis  and  causal  modeling. 


4.2     FACE-TO-FACE  INTERVIEWS 

NIST  selected  ATLAS. ti  as  the  software  system  for  qualitative  analysis.  NuStats  provided  assistance  to 
NIST  in  locating  and  managing  a  qualitative  research  expert  to  support  its  qualitative  analyses. 
Subcontractors  for  this  effort  were  the  co-directors  of  the  survey  laboratory  of  the  University  of  Chicago 
for  the  National  Opinion  Research  Center  (NORC),  Dr.  Martha  van  Haitsma  and  Dr.  Virginia  Bartot. 
These  two  individuals  provide  support  to  NIST  in  developing  the  data  analysis  plan  and  coding  scheme, 
as  well  as  in  ATLAS.ti  software  usage.  After  interview  data  cleaning  and  editing,  which  was  done  in 
Excel,  the  face-to-face  interview  data  was  converted  to  conform  to  the  specifications  of  the  ATLAS.ti 
software.  NuStats  prepared  and  delivered  face-to-face  interview  data  on  a  continuous  basis  throughout 
the  data  collection  period. 


4.3     FOCUS  GROUP  INTERVIEWS 


Focus  group  interview  summaries  were  delivered  to  NIST  as  documents  in  Microsoft  Word.  Data 
confidentiality  constraints  prevented  audiotaping  or  videotaping  of  the  focus  group  sessions.  Thus, 
summaries  were  prepared  by  the  focus  group  facilitator  from  detailed  notes  provided  by  obser\'ers  at  the 
groups. 
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Appendix  A 
Work  Plan 


This  document  provides  documentation  of  the  NuStats  work  plan,  following  a  kick-off  meeting  with 
NIST  staff  and  three  outside  experts  on  June  9-10,  2003.  This  work  plan  updates  both  the  NlST-produced 
white  paper,  "Collection  Methodology  for  Work  Trade  Center  Evacuation  and  Emergency  Response: 
Telephone  Interviews,  Face-to-Face  Interviews,  Focus  Groups  and  Population  Sampling"  (included  as 
part  of  the  original  contract  solicitation),  and  the  Scope  of  Work  (task-by-task  description)  that  was 
included  in  NuStats  proposal. 

A.1  BACKGROUND 

The  goal  of  the  National  Institute  of  Standards  and  Technology's  World  Trade  Center  Investigation  is  to 
investigate  the  building  construction,  the  materials  used,  and  the  technical  conditions  that  contributed  to 
the  outcome  of  the  World  Trade  Center  (WTC)  disaster.  The  results  of  the  Investigation  will  serve  as  the 
basis  for  improvements  in  the  way  buildings  are  designed,  constructed,  maintained,  and  used;  improved 
tools,  guidance  for  industry  and  safety  officials;  revisions  to  codes,  standards,  and  practices;  and 
improved  public  safety.  The  primary  objectives  of  the  NIST-led  technical  investigation  of  the  WTC 
disaster  are  to: 

1 .  Determine  why  and  how  WTC  1  and  2  collapsed  following  the  initial  impacts  of  the  aircraft 
and  why  and  how  WTC  7  collapsed; 

2.  Determine  why  the  injuries  and  fatalities  were  so  high  or  low  depending  on  location, 
including  all  technical  aspects  of  fire  protection,  occupant  behavior,  evacuation,  and 
emergency  response; 

3.  Determine  what  procedures  and  practices  were  used  in  the  design,  construction,  operation, 
and  maintenance  of  WTC  1,  2,  and  7;  and 

4.  Identify,  as  specifically  as  possible,  areas  in  building  and  fire  codes,  standards,  and  practices 
that  are  still  in  use  and  warrant  revision. 

The  NIST  Investigation  Plan  can  be  found  at  http://wtc.nist.gov,  including  a  description  of  Projects  7 
and  8.  Under  Project  7,  Occupant  Behavior,  Egress,  and  Emergency  Communications,  first-hand 
accounts  of  the  events  of  September  11,  2001,  from  inside  WTC  1,  2,  and  7  will  be  collected.  This  data 
collection  effort  will  evaluate  the  role  of  occupant  behavior  and  evacuation  technologies  and  practices  for 
tall  buildings,  including  decision-making  and  situation  awareness,  time-constrained  evacuation  strategies, 
communications,  role  of  floor  wardens  and  fire  safety  directors,  and  issues  concerning  people  with 
disabilities.  Additionally,  NIST  will  seek  specific  observations  of  fire  and  smoke  conditions  and/or 
structural  damage  from  within  the  building.  Families  of  the  victims,  who  communicated  with  loved  ones 
inside  the  towers  before  collapse,  will  be  interviewed  to  determine  the  nature  of  the  environment  above 
the  floors  of  impact.  The  objectives  of  Project  8,  Fire  Service  Technology  and  Guidelines,  are  to  build 
upon  work  already  done  by  the  Fire  Department  of  New  York  (FDNY)  and  McKinsey  &  Company  by: 
(1)  fully  documenting  what  happened  during  the  response  by  the  fire  services  to  the  attacks  on  the 


NISTNCSTAR  1-7B,  WTC  Investigation 


21 


Appendix  A 


World  Trade  Center,  up  to  the  time  of  collapse  of  WTC  7;  (2)  identifying  issues  that  need  to  be  addressed 
in  changes  to  practice,  standards  and  codes;  (3)  identifying  alternative  practices  and/or  technologies  that 
may  address  these  issues;  and  (4)  identifying  R&D  needs  that  advance  the  safety  of  the  fire  service  in 
responding  to  massive  fires  in  tall  buildings.  Thus,  a  subset  of  the  emergency  responders  who  were 
present  at  the  WTC  complex  will  be  asked  to  voluntarily  participate  in  the  face-to-face  interview  or  focus 
group  phases  of  this  project.  Only  first  responders  who  participated  in  fire  suppression,  operational,  or 
search  and  rescue  activities  prior  to  the  building  collapse  will  be  considered  for  inclusion  in  these  phases 
of  the  study. 

A.2         OVERVIEW  OF  METHODOLOGICAL  APPROACH 

The  WTC  interviews  are  being  conducted  to  assist  the  NIST-investigation  team  in  discovering  "what 
happened"  from  the  time  of  the  first  airplane  hit  until  the  collapse  of  the  buildings  (WTC  1,  WTC  2,  and 
WTC  7).  The  interviews  are  focused  on  behaviors  and  observations  that  took  place  within  the  buildings 
and  up  to  a  block  radius  surrounding  the  buildings.  The  interviews  are  not  concerned  with  events 
subsequent  to  the  collapse  of  the  buildings  or  outside  of  this  spatial  radius.  The  data  collection  objective 
is  to  conduct  800  telephone  interviews  with  occupants  of  WTC  1  and  WTC  2;  up  to  600  face-to-face 
interviews  with  occupants  of  WTC  1,  WTC  2,  and  WTC  7,  30  face-to-face  interviews  with  families  of 
victims,  and  150  face-to-face  interviews  of  first  responders;  and  10  focus  group  sessions  with  first 
responders  and  five  (5)  focus  group  sessions  with  building  occupants.  The  types  of  information  to  be 
derived  from  these  data  collection  activities  include: 

•  Identifying  physical  and  human  impedances  and  facilitators  to  building  evacuation, 

•  Developing  a  better  fundamental  understanding  of: 

o  Egress 

o  Human  behavior 

o  Emergency  communications 

o  Emergency  response  procedures 

o  Building/  response  technology. 

This  multi-methodological  approach  was  selected  to  uncover  the  required  information  for  several  reasons. 
First,  multiple  methodologies  increase  confidence  in  the  conclusions  and  findings  when  more  than  one 
methodology  arrives  at  the  same  conclusions.  Second,  the  varied  objectives  of  the  Investigation  mandate 
complementary  approaches  to  accomplish  all  the  goals.  Finally,  concerns  associated  with  the  time  latency 
since  September  11,  2001,  suggested  the  use  of  different  approaches  and  techniques  in  order  to  increase 
memory  recall  and  accuracy.  ^ 
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A.3         TASK  1 :  INSTRUMENT  DEVELOPMENT,  PROTOCOLS  AND  DATABASE 
DESIGN 

A.3.1        Telephone  Interview:  Survey  Instrument 

One  of  the  data  collection  methodologies  is  the  telephone  interview.  The  collection  mechanism  utilized 
in  this  phase  of  the  study  will  be  a  computer-assisted  telephone  interview.  It  is  expected  interview  will 
range  from  20  -  30  minutes,  with  an  average  length  of  25  minutes.  The  primary  goal  of  the  telephone 
interview  is  to  provide  statistical  estimates  that  will  inform  both  the  NIST  Investigation  and  evacuation 
theory,  particularly  as  it  pertains  to  disasters  involving  fires.  The  categories  of  information  to  be  captured 
in  the  telephone  interviews  include: 

•  Means  of  first  alert  (i.e.,  emergency  notification) 

•  Situational  awareness 

•  Factors  associated  with  evacuation  decision 

•  Delay  period  and  activities  associated  with  delay  in  evacuation 

•  Evacuation  route  choice  and  reason  for  choice 

•  Positive  and  negative  aspects  of  egress 

•  Pre-existing  injuries  or  injuries  incurred  during  evacuation 

•  Occupant  characteristics  and  traits 

•  Employer-related  information  relevant  to  the  investigation,  such  as  existence  or  provision  of 
emergency  training 

•  Physical  building  elements  and  human-related  impedances  or  facilitators  encountered  during 
evacuation. 

NuStats  will  use  information  obtained  during  the  kick-off  meeting  to  draft  the  telephone  interview 
instrument.  Such  infonnation  includes  verbal  communications  with  NIST  staff  and  the  outside  experts, 
the  documents  identifying  key  questions  of  the  Investigation  by  project,  the  document  developed  by  E. 
Kuligowski  on  Sample  WTC  Questions,  the  notes  from  the  brainstorming  session  of  the  WTC  project 
leaders  and  team  members  pertaining  to  survey  content,  and  questionnaires  provided  by  G.  Proulx  for 
prior  surveys  on  this  topic.  The  questions  will  flow  in  a  logical  order  in  relation  to  the  chronology  of  the 
events,  as  suggested  in  the  literature. 

NuStats  will  follow  standard  telephone  interview  construction  techniques.  The  first  draft  of  the  telephone 
interview  instrument  will  be  reviewed  in  a  joint  meeting  with  NIST  staff  and  the  outside  experts  and 
subsequently  revised.  Prior  to  this  joint  meeting,  our  consultants.  Dr.  Jon  Krosnick  and  Jamie  Abelson. 
will  review  the  questionnaire  for  content,  flow,  and  question  construction  and  validity.  Once  the  final 
draft  instrument  has  been  developed,  NuStats  (Heather  Contrino)  will  assist  NIST  in  submitting  the 
instrument  package  for  0MB  approval. 
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A.3.1.1      Telephone  Interview:  Survey  Population 

The  persons  who  were  in  WTC  1  and  WTC  2  immediately  prior  to  the  first  aircraft  impact  on 
September  11,  2001,  will  constitute  the  population  to  be  sampled  in  this  study  segment.  The  sampling 
plan  of  the  September  11,  2001,  occupants  is  a  stratified  probability  sample  of  building  occupants. 

Stratifying  the  population.  The  proposed  stratification  criteria  are  based  on  the  assumption  that 
occupants  experienced  unique  egress  behavior,  observed  different  aspects  of  the  structural  integrity  of  the 
buildings  and  suppression  systems,  and  had  varied  interactions  with  first  responders  based  on  their 
location  in  the  buildings  at  the  time  of  the  airline  impacts.  The  capture  of  these  unique  circumstances, 
observations  and  experiences  are  critical  for  NIST  in  understanding  the  egress  process  and  the  structural 
integrity  of  the  buildings,  its  suppression  protocols,  and  first  responder  impact  leading  up  to  the  time  of 
collapse. 

A  primary  stratification  criterion  will  be  building  of  occupancy  -  namely,  WTC  1  and  WTC  2.  A 
secondary  stratification  criterion  will  be  location  within  the  towers  as  defined  by  three  zones  that  reflect 
distance  from  ground  floor  and  proximity  to  point  of  impact.   The  zones  are  defined  according  to  the 
location  of  the  mechanical  floors.  Separate  zones  approximately  comprise  thirds  of  the  buildings: 

•  the  top  floors  (floors  77  to  91  in  WTC  1  and  floors  77  to  1 10  in  WTC  2), 

•  the  middle  floors  (floors  43  to  74  in  WTC  1  and  WTC  2),  and 

•  the  lower  floors  (floors  9  to  40  in  WTC  1  and  WTC  2). 

A  third  stratification  criterion  is  tenant  size.  The  tenant  size  criterion  represents  a  floor  as  one  of  two 
levels: 

•  large  tenant  floor  (a  single  tenant  occupies  greater  than  40  percent  of  the  usable  square 
footage  of  a  floor),  and 

•  small  tenant  floor  (all  other  tenant-occupied  floors). 

Enumerating  the  population.  At  this  time,  NuStats  expects  to  use  the  security  badge  list  as  the  sampling 
frame.  The  list  would  be  sorted  to  approximate  the  stratification  criteria  noted  above.  The  utihty  of  this 
list  is  pending  further  investigation  of  the  completeness  and  reliability  of  the  contact  information.  If  this 
list  is  used,  it  will  constitute  the  sampling  frame  of  WTC  1  and  WTC  2  occupants. 

This  list  is  highly  attractive  because  of  its  full  coverage  of  the  September  1 1  WTC  occupants.  The 
downside  of  the  list  is  that  it  also  contains  a  large  number  of  occupants  who  were  not  in  the  WTC  on  the 
day  of  the  tragedy.  The  list  is  unable  to  distinguish  who  was  present  in  the  WTCs  on  September  11.  This 
means  that  considerable,  unanticipated  screening  will  be  needed  to  identify  September  1 1  occupants. 

Ideally,  security  badge  list  will  contain  home  address  and  telephone  number  as  of  September  11.  If  it 
does  not,  then  considerable  unanticipated  resources  will  be  needed  to  locate  the  subjects.  Even  with  the 
September  11  addresses  and  telephone  numbers,  we  anticipate  that  a  high  proportion  of  subjects  will  have 
moved  or  changed  telephone  numbers  and  this  will  require  an  unanticipated  incremental  locating  effort. 
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Screening.  It  is  important  to  note  that  telephone  screening  will  be  needed  to  determine  whether  an 
individual  on  the  list  is  eligible  -  that  is,  whether  he/she  was  an  occupant  of  WTC  1  or  2  at  the  time  of  the 
first  aircraft  impact  on  September  11,  2001.  It  is  estimated  that  a  total  of  10,000  to  14,000  people  were 
inside  WTC  1  and  WTC  2  at  the  time  of  the  first  impact. 

Selecting  the  sample.  Stratified  probability  sampling  of  individuals  will  be  conducted  from  the  frame  to 
yield  a  total  of  800  telephone  interviews.  Oversampling  of  certain  subgroups  is  expected,  but  the 
delineation  of  those  groups  has  not  yet  been  finalized.  Oversampled  groups  might  include: 

•  occupants  near  the  point  of  impact 

•  occupants  in  each  tower  (e.g.,  equal  allocation  to  WTC  1  and  WTC  2) 

•  others  to  be  specified. 

It  is  assumed  that  there  will  be  an  approximate  30  percent  participation  rate  among  those  asked  to 
participate.  A  final  data  set  of  800  cases  will  ensure  a  0.05  level  of  significance  and  power  of  0.80. 
NuStats  will  make  every  effort  to  increase  participation  above  30  percent. 


A.3.1.2     Telephone  Interview:  Pilot  Testing 

For  the  CATI  pilot,  NuStats  will  randomly  select  10  building  occupants  from  the  list  provided  and 
attempt  to  interview  them.  Our  subcontractor,  DataSource,  will  conduct  these  interviews.  The  Task 
Manager,  Heather  Contrino,  will  be  on-hand  to  directly  interact  with  interviewers  during  the  pilot  phase. 
DataSource  has  technology  for  remote  monitoring  of  interviews  so  that  NIST  staff  can  assess  the 
interview  administration,  if  requested.  All  aspects  of  the  data  collection  plan  will  be  tested  including: 

•  Instrument  flow; 

•  Respondent  comprehension; 

•  Interviewer  administration; 

•  Identification  of  questions  needing  probes  or  clarifications; 

•  Data  coding  and  processing;  and 

•  Review  of  survey  data. 

A.3.2        Face-to-Face  Interviews:  Survey  Instrument 

The  objectives  of  the  face-to-face  interview  segment  is  to  gather  richness  of  detail  about  the  human 
evacuation  experience,  to  obtain  first-hand  accounts  and  observations  of  the  activities  and  events  inside 
the  buildings  on  the  morning  of  September  1 1  for  specialized  investigatory  topics,  and  to  capture  the 
views,  experiences,  and  first-hand  accounts  from  sub-populations  of  high  interest  to  the  Investigation.  It 
is  estimated  that  the  average  face-to-face  interview  will  last  approximately  two  hours,  with  some  lasting 
significantly  longer. 
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The  face-to-face  instrument  design  is  guided  by  the  proposed  methodology,  which  is  a  synthesis  of  the 
Behavioral  Sequence  Interview  Technique  (BSIT)  originally  developed  by  Keating  and  Loftus  and  the 
Cognitive  Interviewing  Method  (CIM),  originally  developed  by  Fisher  and  Geiselman.  These  two 
interviewing  methodologies  were  developed  with  the  purpose  of  assisting  persons  in  retrieving  more 
comprehensive  and  accurate  memories  of  incidents,  and  sharing  important  attributes.  Both  approaches 
begin  by  allowing  the  informant  to  retell  an  unimpeded  account  without  interruption  from  the  interviewer, 
and  both  initially  employ  a  chronological  retelling  of  information.  However,  BSIT  was  designed  to  yield 
a  database  of  qualitative  infonnation  that  could  be  subjected  to  systematic  analysis  and  consolidation, 
while  CIM  was  designed  to  facilitate  investigative  interviews.  Since  the  Investigation  is  pursuing  both 
goals  (i.e.,  creation  of  a  database  of  evacuation-related  behaviors  and  an  investigatory  attempt  to  capture 
information  relevant  to  outcomes),  the  proposed  methodology  combines  these  two  approaches. 

Cognitive  interviewing  has  been  the  subject  of  many  empirical  investigations.  Fisher,  et  al  summarized 
these  findings,  demonstrating  that  the  methodology  significantly  increases  the  amount  of  information 
recalled  without  affecting  rate  of  errors.  Interviewing  a  large  number  of  informants  will  allow 
corroboration  of  infonnation,  thereby  compensating  for  the  likely  increase  in  the  absolute  number  of 
errors.  Accordingly,  it  is  likely  that  this  approach  will  be  productive  in  achieving  a  holistic  view  of  the 
building  evacuations. 

NuStats  will  design  three  paths  within  the  face-to-face  questionnaire:  one  for  occupants,  one  for  family 
members  of  the  victims  who  communicated  with  loved  ones  inside  the  towers  before  collapse  and  one  for 
first  responders.  Depending  on  initial  screening  questions,  the  respondent  will  proceed  through  a 
specialized  version  of  the  questionnaire  directed  at  their  survey  population.  This  will  increase  design 
efficiency  and  allow  all  responses  to  be  stored  in  one  database. 

The  occupant  questionnaire  will  be  used  primarily  to  capture  the  egress  process,  including  occupant 
behavior  and  emergency  communications  in  rich  detail.  The  face-to-face  questionnaire  for  family 
members  will  collect  data  that  describe,  define  and  measure  the  nature,  extent  and  timing  of 
communications  between  family  members  of  the  victims  and  their  loved  ones  inside  the  towers  before 
collapse.  The  first  responder  questionnaire  will  collect  data  on  emergency  communications,  emergency 
response  procedures,  and  building  /  response  technology.  As  with  the  CATI  instrument.  Dr.  Jon  Krosnick 
and  Jamie  Abelson  will  review  the  questions  to  ensure  they  will  meet  the  research  objectives  and,  at  the 
same  time,  minimize  unit  and  item  non-response. 

A.3.2.1      Face-to-Face  Interviews:  Computer-Assisted  Personal  Interview  (CAPI)  Format 

NuStats  will  automate  the  interview  structure  to  increase  efficiency  and  reduce  error.  The  specific  CAPI 
technology  recommended  is  the  use  of  Tablet  PCs.  By  using  this  state-of-the  art  technology,  high  quality 
survey  data  can  be  electronically  coded  and  organized,  and  stored  directly  in  an  electronic  format,  with  all 
relevant  error  checking  performed  automatically.  This  process  will  simultaneously  eliminate  unnecessary 
data  entry  costs  and  associated  key  entry  errors  that  typically  occur  after  a  traditional  paper-based  field 
survey  is  complete.  In  fact,  through  the  use  of  a  Tablet  PC  for  survey  administration,  the  entire  survey 
can  be  captured  and  coded  in  real  time  making  data  delivery  faster  and  easier. 

Tablet  PCs  offer  laptop  PC  functionality  and  processing  power  -  in  a  fonn  factor  that  is  significantly 
smaller  than  a  laptop.  An  example  of  a  Tablet  PC  can  be  seen  in  Figure  A-1. 
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Figure  A-1.  Example  pen  tablet  PC. 

This  pen  tablet  PC  weighs  only  3.1  pounds,  has  a  10.4-inch  viewable  screen,  and  comes  standard  with  an 
800  MHz  CPU,  128  MB  of  RAM,  and  a  20  GB  hard  drive.  In  addition  to  the  standard  touch  screen 
display,  the  latest  generation  of  Tablet  PCs  includes  a  wireless  keyboard  to  facilitate  data  entry.  The 
Tablet  PC  shown  in  Figure  1  converts  easily  from  a  Tablet  PC  to  a  Notebook  PC  form  factor.  Our 
proposed  instrument  design  using  this  technology  is  described  in  the  following  paragraphs.  The 
instrument  will  be  developed  (i.e.,  programmed)  to  collect  the  fact-to-face  interview  data  in  three  stages, 
following  the  steps  of  the  combined  BSIT  /  CIM  interview  process. 

Step  1:  Unimpeded  open-ended  narrative  account.  Both  BSIT  and  CIM  begin  the  process  by  asking 
the  participant  to  chronologically  recount  his  or  her  story.  The  proposed  starting  point  is  when  it  became 
apparent  that  something  unusual  had  occurred  on  the  morning  of  September  1 1,  2001.  The  proposed 
ending  point  is  when  the  participant  feels  that  he  or  she  reached  a  location  where  they  felt  safe  (or, 
alternatively,  when  he  or  she  successfully  reached  the  exterior  of  the  building).  Researchers  and 
practitioners  involved  with  cognitive  interviewing  beheve  that  starting  the  face-to-face  interviews  in  this 
manner  both  improves  recall  and  helps  build  rapport  between  the  participant  and  the  interviewer. 

NuStats  will  expand  on  the  guidehnes  for  step  1  by  organizing  the  interview  process  using  the  sensory 
approach.  Respondents  will  be  asked  something  like.-  I'd  like  you  to  share  your  evacuation  experience 
with  me  from  the  time  you  first  became  aware  that  there  was  something  unusual  occurring  until  you  were 
safety  out  of  the  building.  I 'm  particularly  interested  in  specific  things  you  saw,  heard,  smelled,  or 
touched  during  your  evacuation. 

The  instrument  for  this  phase  is  a  list  of  sensory  experiences  that  the  respondent  may  mention  in  their 
narrative.  The  interviewer  only  "taps"  the  screen  to  record  that  a  particular  type  of  sensory  experience 
was  mentioned.  For  example,  the  participant  might  briefly  mention  an  odd  odor  which  the  interviewer 
makes  note  of  During  the  negotiated  phase  (step  2  below)  the  interviewer  will  want  to  prove  this  fact  to 
determine  whether  the  smell  might  have  been  that  of  jet  fuel,  smoke,  or  of  some  other  origin  as  yet 
unknown. 
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Step  2:  Structured  (negotiated)  narrative  account.  After  participants  complete  their  stories, 
interviewers  will  prompt  them  to  go  through  the  story  again,  but  this  time  they  will  work  cooperatively 
with  the  interviewer  to  record  entries  into  a  table.  This  approach  is  employed  by  BSIT  for  three  primary 
reasons:  (1)  to  yield  a  structured  account  that  can  be  entered  into  a  database  without  further  processing; 
(2)  to  avoid  the  biasing  effects  of  having  interviewers  ask  specific  questions;  and,  (3)  to  enhance  the  effort 
at  recall  put  forward  by  participants  by  encouraging  their  active  collaborative  participation,  an  advantage 
to  open-ended  formats  as  noted  by  Fisher,  et  al. 

Each  row  of  the  table  will  represent  a  single  action  in  a  sentential  format,  meaning  that  each  action  is 
expressed  as  a  grammatical  sentence.  The  approach  is  based  on  the  hypothesis  that  people  encode 
episodic  memories  in  a  manner  consistent  with  this  format,  thus  facilitating  both  recall  and  data  entry. 
Each  coluinn  of  the  table  represents  three  essential  components  of  actions:  a  cue,  an  action,  and  the  reason 
for  taking  that  action  (see  Table  A-1). 


Table  A-1.  Example  tabular  face-to-face  interview  data  entry. 


Cue 

Action 

Reason 

1  heard  but  couldn't  see  someone 
yell  "I've  found  a  clear  path" 

So  1  stumbled  in  the  dark  towards 
where  1  thought  the  voice  came 

So  that  I  could  find  a  way  to 
escape 

My  path  was  blocked  by  debris 

So  ]  called  out  to  whoever  yelled, 
"I'm  near  the  reception  area. 
Where  are  you? 

To  try  to  get  a  better  idea  of 
where  the  person  was 

Cues  can  be  either  external  (e.g.,  signs  of  a  fire,  someone  saying  something)  or  internal  (e.g., 
remembering  about  another  means  of  escape).  For  purposes  of  this  research,  cues  will  be  sensory 
experiences.  Actions  are  expressed  using  specific  action  verbs  (i.e.,  ran.  instead  of  went)  and  may  include 
artifacts  (e.g.,  a  fire  extinguisher)  used  by  the  infonnant.  Reasons  are  the  intentional,  goal-directed  base 
for  the  action.  The  interviewer  will  encourage  the  participants  to  use  their  own  words  to  the  greatest 
extent  possible. 

Experimental  findings  in  psychological  research  on  memory  suggest  that  when  people  perform  actions, 
their  abilities  to  verbally  recall  those  actions  are  significantly  improved.  Script  theory  suggests  that 
people  naturally  organize  their  knowledge  of  actions  using  narrative  sequences  of  actions  structured 
around  their  pursuit  of  goals.  However,  gaps  in  the  narrative  are  anticipated,  especially  given  the  long 
period  of  time  that  will  have  elapsed  between  the  event  and  the  interview.  The  information  entered  by  the 
interviewer  in  Step  1  on  the  sensory  experiences  recounted  in  the  unstructured  narrative  will  be  used  to 
"populate"  a  table  prior  to  the  start  of  the  negotiated  structured  narrative  report.  This  pre -populated  table 
will  be  used  to  structure  the  re-telling  of  the  respondent's  story.  In  addition,  interviewers  will  assist  the 
participants  to  fill  in  these  gaps  by  asking  them  to  recall  events  in  reverse  order,  an  approach  used  in 
CIM.  Interviewers  will,  however,  encourage  participants  to  report  only  those  memories  about  events  or 
incidents  that  they  are  confident  really  occurred  to  them. 

Step  3:  Probing  for  specific  information.  After  completing  the  negotiated,  structured  narrative  account, 
interviewers  will  ask  specific  open-ended  questions  (probes)  intended  to  provide  specific  information  of 
particular  value  to  the  investigation.  While  some  of  this  infonnation  is  likely  to  be  part  of  the  structured 
narrative  account,  participants  may  be  able  to  recall  other  valuable  information  as  well.  In  this  step, 
interviewers  will  ask  questions  related  to  topics  that  have  been  identified  a  priori  as  being  priority  topics. 
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This  will  ensure  that  all  information  of  significant  importance  to  the  Investigation  will  be  examined  in  the 
face-to-face  portion. 

Depending  on  the  population,  probes  may  be  used  to  try  to  elicit  information  including,  but  noi  linuicd  lo: 

•  Location  of  the  informant  at  the  time  of  certain  marker  events  (e.g.,  location  in  WTC  1  when 
WTC  2  collapsed) 

•  Fire  conditions  (e.g.,  fire  and  smoke); 

•  Other  cues  of  interest  (e.g.,  the  smell  of  jet  fuel); 

•  Presence  and  activities  of  persons  with  disabilities; 

•  Use  of  elevators  by  self  or  others;  and, 

•  Knowledge  of  any  obstacles  to  their  progress  while  using  the  stairs. 

Because  information  about  many  of  these  areas  of  concern  requires  precise  responses,  questions  for  open- 
ended  probes  will  be  developed  collaboratively  between  NuStats  and  NIST.  Responses  to  probes  may  be 
recorded  using  standardized  formats  where  feasible.  For  example,  all  participants  who  observed  smoke 
may  be  asked  to  estimate  the  smoke  density  using  an  encodable  scale,  such  as  visibility  distance.  To  help 
standardize  scales  or  indexes  used  and  also  the  reporting  of  locational  information,  graphical  prompts  will 
be  included  in  the  software  to  allow  for  better  specification. 

A.3.2.2     Face-to-Face  Interviews:  Survey  Population 

The  populations  to  be  sampled  in  this  study  segments  include  the  following  types  of  people: 

•  individuals  who  were  in  WTC  1,  WTC  2,  and  WTC  7  immediately  prior  to  the  first  aircraft 
impact  on  September  1 1 ,  2001 ; 

•  individuals  who  communicated  with  members  of  their  families  who  were  in  WTC  1,  WTC  2, 
and  WTC  7  immediately  prior  to  the  first  aircraft  impact  on  September  11,  2001,  and  who 
never  made  it  out  of  the  buildings  safely; 

•  individuals  who  were  among  the  first  responders  (i.e.,  firefighters  and  police)  to  the  WTC 
disaster. 

The  sample  will  be  selected  on  the  basis  of  NIST  knowledge  of  populations  that  may  have  unique  pieces 
of  information  to  add  to  the  Investigation.  These  are  persons  who  belong  to  special  subpopulation  groups 
or  were  in  locations  during  the  evacuation  that  enabled  them  to  experience  distinctive  situations.  This 
method  of  sampling  is  called  purposive  or  judgmental  sampling.  The  populations  of  interest  for  the  face- 
to-face  interviews  are: 

•  Occupants  near  the  floors  of  impact  (WTC  1  and  WTC  2) 

•  Occupants  of  WTC  7 
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•  Floor  wardens,  fire  safety  directors,  and  other  persons  with  building  responsibility 

•  People  in  elevators  or  lobbies 

;        •    Occupants  with  disabilities  - 

•  Family  members  of  victims  who  called  out  of  the  towers 

•  First  responders  (members  of  the  Fire  Department  of  New  York,  Port  Authority  Police 
Department,  New  York  Police  Department,  and  other  having  operational  responsibility. 

While  the  exact  numbers  of  interviews  for  each  group  has  not  been  finalized,  the  total  number  of  face-to- 
face  interviews  will  be  750  intei'views,  of  which  150  will  be  with  first  responders. 

Enumerating  the  population.  The  population  will  include  the  entire  occupant,  management,  and  first 
responder  population  of  World  Trade  Center  WTC  1,  2,  and  7. 

Selecting  the  sample.  The  sample  for  the  face-to-face  interviews  is  purposive.  The  potential 
respondents  will  be  located  via  a  number  of  methods.  For  example,  NIST  will  provide  the  names  and 
contact  information  for  the  first  responder  sample.  NuStats  expects  to  receive  approximately  300  names 
from  which  to  select  these  respondents.  Other  special  populations  can  be  identified  through  media 
accounts.  NuStats  will  work  with  Guylene  Proulx  to  identify  potential  respondents  as  she  is  the 
contractor  for  another  WTC-related  study  of  media  coverage.  A  number  of  the  respondents  for  the  face- 
to-face  interviews  will  be  identified  via  the  CATI  interviews.  During  CATI  interview,  it  will  be  noted 
when  a  respondent  has  characteristics  or  experiences  matching  the  subpopulations  of  interest.  In  such 
cases,  the  respondent  will  be  interviewed  via  CATI  and  then  an  appointment  will  be  set  up  for  a  face-to- 
face  interview. 

A.3.2.3     Face-to-Face  Interviews:  Pilot  Testing 

The  pilot  of  the  face-to-face  instrument  will  be  done  in  New  York  City  at  MBC  Research  Center.  Johanna 
Zmud  (project  director),  Robert  Santos  (senior  methodologist),  and  Delia  Santos  (task  manager),  will 
conduct  the  pilot  face-to-face  interviews.  A  total  of  five  face-to-face  interviews  will  be  conducted  among 
occupants,  family  members  and  /  or  first  responders.  The  viewing  room  at  MBC  Research  will  enable 
NIST  staff  to  observe  the  pilot  interviews.  As  with  the  CATI  interviews,  all  six  (6)  aspects  of  data 
collection  will  be  tested. 

A.3.3        Focus  Groups:  Interview  Protocol 

The  goal  of  the  focus  group  interviews  is  to  elicit  accurate  group  representations  of  specific  events  or 
themes.  Williams  reports  that  in  a  group  setting,  people  provide  cues  that  evoke  memories  in  others,  and 
that  social  pressures  mediate  against  reporting  misrepresentations  of  what  they  recall.  The  interview 
protocols  will  build  upon  the  concepts  and  operationalizations  used  in  the  CATI  and  face-to-face 
instruments.  Focus  group  protocols  will  be  designed  to  help  facilitate  recall  and  elicit  group 
representations  of  specific  events,  communications,  and  egress  behavior.  This  will  be  accomplished 
through  a  tiered  focus  group  protocol  that  begins  with  less  sensitive  topics  and  progresses  into  more 
detailed  and  difficult  information  as  the  session  proceeds. 
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A.3.3.1      Focus  Groups:  Target  Population 

Two  distinct  populations  will  voluntarily  participate  in  the  focus  groups:  occupants  and  first  respondcrs. 
The  first  set  of  focus  group  interviews  will  be  the  occupant  sample.  Distinct  categories  of  people  will  be 
selected  for  inclusion  in  this  study.  The  objective  of  this  study  is  to  capture  the  experience  of  people  in 
unique  places  in  WTC  1,  2,  and  7.  Some  of  these  groups  have  been  identified  by  NIST:  the  1 6  people 
who  were  above  floor  78  who  made  it  safely  out  of  WTC  2,  disabled  persons,  persons  who  saw  the  lobby 
fireball,  and  persons  who  were  trapped  in  the  elevator.  Every  effort  will  be  made  to  include  no  less  than 
5  people  in  each  of  these  categories  in  this  study,  with  10  people  constituting  the  preferred  focus  group 
size.  NuStats  anticipates  conducting  five  occupant  focus  groups. 

First  responders  will  constitute  a  second  set  of  focus  group  interviews.  The  set  of  first  responders  will 
include  FDNY,  NYPD,  PAPD,  and  other  groups  identified  as  having  operational  or  command  authority  at 
the  World  Trade  Center  on  September  1 1th.  The  focus  group  size  will  be  determined  as  an  operating  unit 
size,  if  applicable.  An  operating  unit  may  be  a  Fire  Department  company,  for  example.  This  project 
proposes  10  focus  groups,  each  containing  five  people. 

Sample  selection.  The  people  selected  for  inclusion  in  the  focus  groups  will  be  selected  using  non- 
probability  sampling  procedures.  Respondents  contacted  for  the  CATI  interviews  or  face-to-face 
interviews  may  be  eligible  for  participation  in  the  focus  groups.  In  addition,  respondents  in  these  other 
studies  will  be  asked  to  provide  the  names  and  contact  information  for  people  they  know  in  each  of  the 
categories  of  interest.  The  contractor  will  collect  names  until  at  least  five  people  in  each  category  have 
agreed  to  participate  in  an  occupant  focus  group,  with  a  preference  for  10  people.  It  is  expected  that  the 
potential  participant  list  for  the  first  responder  focus  groups  will  be  provided  by  NIST.  The  same  process 
will  occur  for  selection  of  the  first  responder  samples,  with  a  preference  for  inclusion  of  entire  operating 
units  (about  5  people  per  unit). 

A.3.4        Instrument  Evaluations 

Each  instrument  will  be  evaluated  prior  to  pilot  testing.  The  instrument  review  process  will  benefit  from 
using  cognitive  interviews  to  identify,  measure,  and/or  reduce  survey  response  errors.  The  administration 
of  this  survey  requires  respondents  to  accurately  report  factual  data  for  which  knowing,  remembering,  and 
placing  events  in  time  are  critical.  Cognitive  interviews  of  likely  respondents  would  serve  to  identity  and 
assess  the  potential  for  survey  response  errors  by  examining  comprehension  of  the  survey  item,  retrieval 
of  relevant  information,  judgment  based  on  recall,  and  mapping  and  reporting  of  a  response. 

Cognitive  testing  is  relatively  new  to  the  survey  industry  (it  has  come  into  use  only  in  the  past  15  years  or 
so).  It  involves  the  observation  and  testing  of  prototype  respondents  in  a  controlled  laboratory  setting  to 
provide  insight  into  the  cognitive  survey  process  that  is  undertaken  by  a  subject  who  is  interviewed  by  an 
interviewer.  The  cognitive  survey  response  process  literally  refers  to  what  happens  in  a  subject's  mind 
between  the  time  he/she  is  asked  a  question,  up  to  and  including  the  point  that  he/she  provides  a  response. 
It  is  composed  of  four  stages:  (a)  comprehension;  (b)  retrieval  of  information;  (c)  response  formation; 
and  (d)  response  editing.  Cognitive  testing  provides  insight  into  these  areas  in  a  way  that  can  lead  to 
significant  improvements  in  question  wording  and/or  validation  of  research  constructs. 

Not  all  questions  in  an  instrument  require  cognitive  testing.  Dr.  Jon  Krosnick  is  an  expert  in  the  cognitive 
survey  process  and  he  will  assist  in  the  identification  of  the  questions  that  do  require  cognitive  testing. 
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Our  approach  would  be  to  identify  in  the  review  process  a  subset  of  critical  questions  and  constructs  that 
might  benefit  from  cognitive  testing  (regardless  of  the  mode  of  administration  -  CAT!,  face-to-face,  or 
focus  groups),  and  to  employ  a  small  number  of  cognitive  interviews  with  likely  respondents  to  address 
them.  Cognitive  tests  will  be  done  with  individuals  known  to  NuStats  who  were  building  occupants  on 
September  11. 

A.3.5        Finalize  Survey  Instruments  and  Interview  Protocols 

After  completion  of  the  pilot  tests,  NuStats  will  provide  a  report  that  details  recommended  changes  in  the 
questionnaire/  protocols  and  training  materials.  This  report  will  include  a  discussion  of  recommendations 
for  modifying  question  order  /  sequencing,  introductory  statements,  response  categories  and  question 
wording  that  might  be  problematic  for  the  respondent.  The  report  will  also  include  a  discussion  of 
recommendations  for  modifying  interviewer  instructions. 

A.3.6        Pre-Field  Work  Meeting 

A  working  session  with  NIST  and  its  outside  experts  will  be  held  in  which  NuStats  details  problems, 
identifies  questionnaire  items  that  need  revision,  and  suggests  corrections  for  problems  encountered 
during  the  pilot.  Following  this  meeting,  NIST  will  notify  NuStats  of  any  recommended  changes  in  the 
survey  instrument  and  protocols.  NuStats  will  incorporate  only  NIST-approved  changes  to  the 
instruments  and  in-person  protocols  before  going  into  the  field. 

A.3.7        Protection  of  Human  Subjects  and  IRB  Approval 

This  data  gathering  effort  will  ensure  that  all  precautions  required  by  the  Common  Rule  for  the  Protection 
of  Human  Subjects  are  met  or  exceeded  by  the  contractor.  Participation  in  any  part  of  this  project  by  any 
person  will  be  strictly  voluntary.  Interviewers  will  be  trained  to  establish  a  rapport  with  participants 
based  on  a  compassionate  interest  in  their  story  and  will  ensure  participants  that  information  provided  will 
be  of  value  in  preventing  casualties  in  future  building  emergencies.  During  the  briefing,  interviewers  will 
provide  information  to  participants  about  where  and  how  to  receive  counseling  and  about  the  fact  that 
participants  may  stop  the  interview  at  any  time  without  explanation.  Interviewers  will  also  be  trained  to 
recognize  signs  of  post-traumatic  stress.  Similar  services  will  be  offered  to  participants  of  focus  groups 
and  to  people  taking  the  telephone  interview.  NuStats  will  take  the  necessary  precautions  to  ensure  the 
safety  of  contract  employees  administering,  collecting,  or  otherwise  involved  in  this  data  collection  effort. 
Finally,  Nustats  will  ensure  that  the  identities  of  the  subjects  are  held  in  the  strictest  confidence. 

The  human  subjects  protocols  to  be  used  in  this  project  by  NuStats  and  DataSource  will  be  reviewed  by 
Essex  Institutional  Review  Board. 

A.3.8        Training  of  Surveyors 

NuStats  will  train  a  team  of  survey  specialists  for  CATI  and  face-to-face  administration  as  well  as  focus 
group  moderators.  Training  protocols  will  be  prepared  by  each  of  the  Task  Managers  (Heather  Contrino, 
Delia  Santos,  Kim  Hilsenbeck)  under  the  direction  of  the  Project  Director,  Dr.  Johanna  Zmud.  All 
interviewers  will  receive  an  eight-hour  training  session  and  will  be  required  to  perform  simulated 
interviews  before  beginning  actual  data  collection  activities.  NuStats  produces  project-specific 
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interviewer-training  manuals  for  each  project.  The  basic  elements  of  the  manual  include  an  overview  of 
the  study,  its  objectives,  glossary  of  ternis,  and  questions  and  answers  typically  posted  by  the  respondent. 
Another  aspect  of  training  deals  with  the  intricacies  of  the  survey  instruments  themselves,  with  separate 
training  for  each  survey  instrument  involved. 

In  this  case,  training  materials  and  sessions  will  include  specific  provisions  to  address  the  unique  needs  of 
this  study  including: 

•  Informed  consent; 

•  Privacy/confidentiality; 

•  Elements  of  risk;  and 

•  Provisions  for  the  protection  of  human  subjects. 

Mock  interviews  are  used  to  help  surveyors  quickly  become  familiar  with  the  survey  and  nuances  of  the 
formatting  and  layout.  The  mock  interviews  will  also  serve  to  help  face-to-face  interviewers  practice 
handling  encounters  with  different  types  of  respondents.  Surveyors  will  not  begin  working  on  this  study 
until  they  have  passed  a  project-specific  test. 

The  telephone  interviewers  will  be  regular  staff  of  DataSource.  Interviewers  for  the  face-to-face 
interviews  will  be  contracted  social  work  practitioners.  Even  though  the  social  workers  will  be  sensitive 
to  context  in  which  the  interviews  are  being  done,  all  interviewers  and  moderators  will  receive  specialized 
training.  Our  consultant,  Jamie  Abelson,  will  be  particularly  active  in  this  training  component.  This 
component  will  include  techniques  for  dealing  with  emotionally  traumatized  respondents,  as  well  as  voice 
tone  and  emotional  mediation  techniques.  The  contracted  social  workers  will  also  be  trained  specifically 
in  BSIT  and  CIM  techniques  using  in-house  experts  and  outside  practitioners. 

CATI  interviewers  will  be  trained  at  DataSource,  NuStats'  subcontractor,  while  face-to-faee  inter\'iewers 
will  be  trained  at  NuStats'  space  in  New  York  City.  Since  focus  group  moderators  are  all  in-house  staff, 
Ms.  Abelson  will  brief  them  at  the  NuStats'  offices  in  Austin  or  in  the  D.C.  area  if  the  NIST  staff  would 
care  to  attend. 

Both  NuStats  key  staff  and  DataSource  supervisors  will  continually  monitor  interviewers  and  moderators 
involved  in  the  study  to  ensure  the  highest  level  of  quality  is  maintained.  Interviewer  debriefings  will  be 
held  weekly  to  address  any  data  collection  issues  that  arise  and  to  heighten  interviewers'  performance. 

It  should  be  noted  that  NuStats  will  require  all  staff  working  on  this  project  to  sign  new  legally  binding 
pledges  of  confidentiality  specific  to  this  project,  in  addition  to  the  pledges  all  staff  sign  as  part  of  their 
employment  contract  with  the  firm.  Any  additional  subcontractors  or  consultants  retained  for  this  project 
will  be  required  to  do  the  same. 

A.3.9        Database  Design 

NuStats  will  design  a  database,  i.e.,  an  encoded  table  of  results,  based  on  the  developed  coding  scheme  of 
all  concepts  and  their  operationalized  measures  used  in  the  telephone  and  face-to-face  approaches.  Thus, 
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the  database  will  consist  of  two  parts:  CATI  data  and  face-to-face  data.  Focus  group  infomiation  will  be 
delivered  to  NIST  as  court  transcripts.  NuStats  will  design  the  database  structures  for  the  telephone  and 
face-to-face  surveys  according  to  the  NIST  specifications.  Upon  final  approval  of  the  questionnaires  and 
moderation  guides  for  each  component  of  the  study,  the  NuStats  team  will  develop  the  database  structures 
and  fully  document  variable  names,  structure,  and  skip  patterns  in  a  comprehensive  Data  Matrix.  One 
data  matrix  will  cover  each  of  the  two  databases.  The  Data  Matrix  will  include  all  variable  specifications 
including  built  in  quality  assurance  measures  (such  as  edit,  range,  and  consistency  checks)  and  will  serve 
as  the  final  road  map  for  database  development.  The  number  of  unique  variables  comprising  the  data 
bases  has  not  be  specified,  however,  the  RFP  states  that  there  will  be  no  more  than  75  encoding  variables. 

To  protect  the  confidentiality  of  participants,  each  data  record  will  be  assigned  a  unique  identifying 
number.  This  will  ensure  that  the  names  of  study  participants  are  not  associated  with  responses  to 
questions.  Moreover,  the  data  file  containing  the  link  between  name  and  ID  number  are  stored  separately 
from  the  data  files  containing  question  responses.  All  confidential  information  will  be  stored  in 
password-protected  files  by  the  holders  of  this  information. 

A.3.10      Additional  Data  Collection 

NuStats  understands  that  the  scenario  may  arise  where  an  individual  critical  to  developing  an 
understanding  of  the  events  of  September  11,  2001,  may  be  unavailable  or  unidentified  during  the  period 
of  performance  of  the  Contractor.  Thus,  NIST  will  obtain  NIST  Institutional  Review  Board  (IRB) 
approval  to  conduct  a  limited  number  of  face-to-face  interviews  with  individuals  deemed  by  NIST  likely 
to  contribute  significantly  to  the  outcome  of  the  Investigation.  The  scope,  objectives,  and  procedures 
used  in  the  additional  data  collection  will  be  similar  to  the  scope,  objectives,  and  procedures  used  by  the 
contractor.  Telephone  interviews  and  focus  groups  will  not  be  conducted  in  this  additional  data  collection 
effort.  It  is  anticipated  that  the  number  of  face-to-face  interviews  conducted  by  NIST  will  be  less  than  10 
percent  of  the  number  of  face-to-face  interviews  conducted  by  the  contractor.  The  contractor  will  incur 
no  duties  or  obligations  related  to  the  additional  data  collection. 

A.4         TASK  2:  SURVEY  ADMINISTRATION 

A.4.1        CATI  Field  Work 

NuStats  subcontractor,  DataSource,  will  reach  a  participation  rate  of  30  percent  to  conduct  a  minimum  of 
800  interviews  with  building  occupants.  Due  to  the  length  and  nature  of  the  interview,  we  will  not 
include  partial  interviews  as  completes. 

DataSource  will  use  a  small,  well-trained  team  of  8-12  interviewers  to  conduct  the  CATI  interviews. 
Each  interviewer  will  be  experienced  in  dealing  with  sensitive  subjects,  well  versed  in  mental  health 
protections  and  confidentiality  issues,  and  will  have  passed  an  extensive  training  exam  before  being 
allowed  to  dial  on  the  survey. 

It  is  expected  that  data  collection  will  last  for  eight  weeks.  This  is  to  allow  for  the  time  it  will  take  to 
reach  a  particular  respondent  (research,  initial  contact,  follow-up  contact/  interview)  and  to  track  those  not 
easily  reached  to  achieve  the  desired  participation  rate.  Dialing  hours  will  be  10  a.m.  to  9  p.m.  (Eastern 
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Time)  and  calling  will  vary  across  days,  nights,  and  weekends  to  maximize  response.  NIST  will  provide 
the  sampling  frame  to  the  NuStats  team  in  the  format  requested  by  NuStats. 

A  phone  call  will  be  made  to  the  sampled  individuals.  NuStats  will  vary  the  day  and  time  of  the  phone 
call  to  increase  the  likelihood  that  contact  with  the  individual  will  be  made.  If  an  individual  chooses  to 
participate,  he/  she  will  be  asked  to  voluntarily  disclose  a  preferred  means  of  follow-up  contact. 
Regardless  of  the  preferred  means,  all  contacted  individuals  will  receive  a  personalized  mailing  that 
details  the  purpose  and  goals  of  the  study,  the  client,  and  the  types  of  the  questions  that  will  be  asked. 
The  mailing  of  the  letters  will  be  done  prior  to  the  expected  day  /  time  of  the  follow-up  contact. 
Individuals  will  be  instructed  to  phone  a  toll-free  number  at  DataSource  should  they  require  further 
information. 

Heather  Contrino,  task  manager,  will  monitor  progress  through  well-established  communications  with  the 
DataSource  field  team.  NuStats  project  managers  receive  nightly  dialing  reports  (documenting  the 
number  of  completed  interviews)  and  weekly  disposition  reports  (a  cumulative  disposition  file  of  all 
numbers  dialed  to  date)  from  DataSource.  Ms.  Contrino  will  customize  this  reporting  to  suit  NIST 
progress  reporting  needs. 

NuStats  is  always  concerned  with  non-response,  and  understands  that  the  response  rate  for  this  sur\'ey  is  a 
critical  element  of  the  data  collection  process.  We  will  use  time-tested  methods  for  minimizing  refusals 
without  infringing  on  the  voluntary  nature  of  this  survey.  Techniques  to  combat  non-response  will 
include: 

•  Advance  notification 

•  The  use  of  Lexis/Nexis  to  locate  "missing"  respondents  selected  for  the  study;  and 

•  A  toll-free  respondent  hotline  and  a  respondent  web  page. 

Advance  notification  has  been  proven  in  studies  such  as  this  one  to  minimize  refusals.  Whenever 
possible,  we  will  send  a  pre-notice  letter  describing  the  survey.  The  letter  will  be  short,  written  in  simple 
language,  will  explain  the  survey  relevance,  distinguish  it  from  other  surveys  investigating  the  WTC 
disaster,  and  will  be  personalized  to  the  extent  possible.  Respondents  will  be  encouraged  to  contact 
DataSource  through  the  respondent  hotline  if  they  have  questions. 

In  some  cases,  contact  information  will  be  outdated  or  nonexistent.  NuStats  anticipates  that  we  may  face 
this  and  other  barriers  in  locating  some  of  the  sampled  occupants.  We  propose  to  use  our  in-house 
resource  Lexis/  Nexis,  for  tracking  subjects  selected  in  the  sample  but  form  whom  inadequate  or  no 
contact  information  is  available. 

A  member  of  our  trained  interviewer  team  will  work  at  a  toll-free  respondent  hotline  during  calling  hours. 
Since  many  respondents  will  have  received  advance  notification,  we  anticipate  that  some  will  have 
questions  or  concerns  about  participating.  Making  staff  available  to  answer  respondent  concerns  is  a 
critical  element  in  building  trust  and  establishing  validity  in  the  survey  design.  NuStats  will  design  and 
informational  (with  Q&A  capabilities)  web  page  for  respondents  who  seek  quick  answers  to  their 
questions. 
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A.4.2        Face-to-Face  Field  Work 

NuStats  will  conduct  750  face-to-face  interviews,  with  occupants,  family  members,  and  first  responders. 
NuStats  will  use  the  synthesized  BSIT  and  CIM  technique  in  conjunction  with  CAPI  technology  to  create 
a  database  of  evacuation-related  behaviors  and  capture  information  relevant  to  outcomes.  NuStats  will 
recruit  the  face-to-face  intei^viewers  from  the  available  pool  of  psychiatric  social  workers  in  the  New 
York  metropolitan  area.  We  expect  this  team  will  be  trained  (and  experienced)  in  dealing  with  sensitive 
topics  and  additional  mental  health  provisions.  Their  experience  will  be  supplemented  with  project- 
specific  training  for  the  WTC  investigation.  This  will  include  training  to  recognize  signs  of  post- 
traumatic stress.  They  will  also  be  able  to  provide  sources  for  additional  mental  health  counseling,  should 
a  respondent  request  it. 

NuStats  expects  to  recruit  a  team  of  8-12  interviewers  and  to  conduct  approximately  6-8  interviews  per 
day  (seven  days  a  week),  with  approximately  2  hours  allocated  for  each  interview.  This  is  a  larger  face- 
to-face  interviewer  pool  than  we  identified  in  the  RFP  to  ensure  that  the  data  collection  in  completed 
within  the  necessary  timeframe.  This  will  require  more  Pen  Tablet  PCs  than  we  had  budgeted. 
Interviews  will  not  be  video-  or  audio-taped  to  protect  the  privacy  and  confidentiality  of  the  respondent. 

NuStats  will  try  to  schedule  the  location  for  the  interviews  to  be  as  convenient  as  possible.  NuStats  will 
employ  a  variety  of  locations  to  ensure  that  a  convenient  location  for  each  face-to-face  respondent  can  be 
achieved.  Locations  include  NIST  space,  MBC  research  space,  space  in  lower  Manliattan  at  181  Mott 
Street,  as  well  as  the  respondent's  home  or  work  place,  if  so  desired. 

Della  Santos,  task  manager,  will  manage  all  fieldwork.  DataSource  will  handle  the  recruiting  and 
scheduling  for  all  face-to-face  interviews.  Because  DataSource  interviewers  will  go  through  intense,  in- 
depth  training  on  the  WTC  project,  we  believe  they  will  be  proficient  in  dealing  with  this  sensitive  topic, 
and  will  be  able  to  best  answer  questions  about  study  objectives  and  how  results  will  be  used. 

A  confirmation  letter  will  be  mailed  to  each  individual  who  agrees  to  participate  in  the  study.  The  letter 
will  explain  the  purpose  of  the  interview,  describe  how  the  interview  will  be  conducted,  confirm  the  time 
and  date  of  the  interview,  and  provide  directions  to  the  interview  location.  Reminder  calls  will  be  made 
to  these  persons  by  the  actual  face-to-face  interviewer  to  whom  the  case  has  been  assigned.  This  will  help 
establish  a  relationship  between  the  intei^viewer  and  respondent  prior  to  the  actual  interview. 

Through  the  use  of  the  CAPI  technology,  data  will  be  entered  and  coded  almost  in  real-time.  This  will 
enable  NuStats  to  provide  data  from  these  interviews  on  an  interim  schedule. 

A.4.3        Focus  Group  Field  Work 

Because  the  focus  groups  are  expected  to  be  longer  than  typical  (approximately  4  hours)  and  emotional  in 
content,  NuStats  will  employ  a  team  of  experienced,  in-house  focus  group  moderators  to  conduct  the 
focus  groups  with  occupants  and  first  responders.  These  moderators  include  Dr.  Carlos  Arce,  Dr. 
Johanna  Zmud,  and  Kim  Hilsenbeck.  Ms.  Hilsenbeck  will  also  manage  this  task,  ensuring  that  all  groups 
are  organized  and  scheduled  as  required. 

DataSource  will  handle  the  recruiting,  scheduling,  and  confirmation  for  all  focus  groups  (for  the  same 
reasons  noted  for  the  face-to-face  interviews).  Focus  groups  will  be  held  at  MBC  Research  Center  on 
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Madison  Avenue.  A  confirmation  letter  will  be  mailed  to  each  respondent  that  explains  the  purpose  of  the 
focus  group  and  the  protocols  in  place  for  identify  protection,  describes  how  the  session  will  be 
conducted,  and  provides  directions  to  the  focus  group  facility.  DataSource  interviewers  will  also  make  all 
the  reminder  calls. 

Each  moderator  will  employ  a  moderator  protocol  for  the  response  population.  Discussion  guidelines 
provide  a  general  framework  for  questioning  and  discussion  during  a  focus  group.  NuStats  will  develop 
this  guideline  in  conjunction  with  NIST  and  the  outside  experts.  While  the  focus  group  moderators 
follow  guides  as  close  as  possible,  each  is  trained  to  understand  that  additional,  relevant  topics  may 
surface  during  the  course  of  a  focus  group  session.  Flexibility  is  critical  to  gaining  the  most  valuable 
insight  from  respondents.  If  participants  diverge  into  topics  that  would  provide  useful  data  for  the  study, 
our  moderators  are  experienced  in  allowing  the  divergence  without  going  too  far  off  course. 

NuStats  anticipates  that  focus  groups  will  last  approximately  four  hours,  with  hourly  breaks  provided. 
The  moderators  will  provide  an  introduction  at  the  outset  of  each  session,  explaining  the  voluntary  nature 
of  respondents'  participation.  This  introduction  will  also  provide  the  basic  ground  rules  for  discussion  as 
well  as  disclose  any  court  transcription  or  viewers.  The  moderator  will  also  relate  the  purpose  of  the 
study  and  how  the  focus  group  results  will  be  used.  NuStats  expects  that  there  may  be  questions  about 
privacy  issues,  and  we  will  use  this  time  to  reassure  all  participants  about  the  high  security  protocols  in 
place  to  protect  the  identity  of  and  information  provided  by  respondents. 

A.4.4        Status  Reporting  and  Data  Deliveries 

Monthly  status  reports  will  be  delivered  to  NIST  to  monitor  progress  in  accordance  with  the  statement  of 
work.  These  reports  will  advise  NIST  of  work  completed  during  the  performance  period,  and  work 
forecasted  for  the  next  performance  period.  NuStats  will  also  provide  data  deliveries  with  these  status 
reports.  Data  deliveries  (for  CATI  and  face-to-face  interviews  and  transcripts  of  focus  groups)  will  be 
delivered  at  the  end  of  September,  October,  and  November. 

NuStats  expects  to  have  a  meeting  with  NIST  and  the  outside  experts  after  the  delivery  of  the  September 
progress  report  and  data  to  review  the  data  and  any  issues  that  have  arisen  with  the  fieldwork. 

A.4.5        Completion  of  Field  Work 

Fieldwork  will  be  deemed  complete  with  NuStats  has  accomplished  all  of  the  sampling  requirements  set 
forth  in  this  finalized  work  plan.  We  expect  the  vast  majority  of  fieldwork  to  be  completed  by  the  end  of 
November.  There  may  be  some  leakage  of  field  work  into  early  December  but  expect  this  to  be  minimal. 

At  the  completion  of  fieldwork,  NIST  will  have  received  the  majority  of  all  required  data  through  the 
interim  data  deliveries.  Three  separate  databases  will  be  present.  Each  data  set  will  be  cleaned  prior  to 
delivery.  If  NIST  finds  any  data  cleaning  issues,  these  will  be  dealt  with  promptly.  To  facilitate  data 
quality  assurance,  NuStats  will  prepare  a  data  quality  assurance  plan  for  NIST  review  prior  to  the  start  of 
fieldwork. 

At  the  completion  of  all  fieldwork,  NuStats  will  prepare  a  summary  analysis  for  NIST  review.  The 
summary  analysis  will  contain  three  parts.  The  first  part  is  a  compilation  of  on-going  (monthly)  statistical 
summaries  of  survey  progress,  consistency  checks,  plausibility  checks,  and  other  indicators  of  data  quality 
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for  the  CATI  and  CAPI  data.  Second,  a  summary  of  the  fielding  for  the  CATI  and  CAPI  survey  response 
will  be  summarized,  including  number  of  contacts,  participation  rates,  completed  responses  in  each  cell  of 
the  sampling  stratification,  completed  response  rates  by  question.  Third  a  descriptive  accounting  of  the 
CATI  and  CAPI  data  will  be  provided,  including  frequency  distributions  and  calculations  of  the  mean  and 
variance  of  the  sample  for  all  data  elements. 

A.4.6  Debrief 

At  the  conclusion  of  data  collection,  NuStats,  NIST  and  the  outside  experts  will  meet  for  a  project  debrief 
This  meeting  will  allow  for  discussion  of  survey  resuhs.  From  NuStats,  participants  will  include,  among 
others.  Dr.  Johanna  Zmud,  Project  Director,  and  each  of  the  Task  Managers  Heather  Contrino,  Delia 
Santos,  and  Kim  Hilsenbeck.  Dr.  Carlos  Arce  and  Robert  Santos  will  also  be  likely  attendees  from 
NuStats.  Prior  to  coming  to  this  debrief  meeting,  each  of  the  Task  Managers  will  have  debriefed 
surveyors  within  their  span  of  control.  They  will  compile  and  bring  to  the  meeting  the  insights  and  other 
feedback  garnered  directly  from  those  involved  in  the  actual  interviewing.  ■ 

A.4.7        Final  Field  Work  Report 

NuStats  will  provide  two  data  sets  -  one  for  the  CATI  data  and  one  for  the  CAPI  data.  Focus  group  data 
will  be  comprised  of  transcriptions  of  the  focus  group  sessions. 

In  addition,  NuStats  will  prepare  a  comprehensive  draft  final  report  summarizing  our  work  under  the 
contract.  This  final  report  will  fiilly  document  the  study  and  sample  design  including  instrument,  sample, 
or  methodological  changes,  result  of  the  cognitive  interviews  and  pilot,  documentation  of  data  collection 
period  and  protocols,  data  processing  procedures,  and  any  conclusions.  Response  rates  and  call 
dispositions  will  also  be  included.  In  addition,  the  report  will  include  all  materials  used  in  the  study 
including  the  CATI  script,  CAPI  script,  focus  group  protocols,  and  data  file  structures. 

NIST  will  have  30  days  to  comment  on  the  draft  final  report.  NuStats  will  incorporate  these  comments 
any  format  requirements  into  the  final  report  and  deliver  four  copies  within  15  days  of  receipt  of  NIST 
comments. 

A.5         TASK  3:  DATABASE  COMPLETION  AND  DELIVERY 

NuStats  will  deliver  the  final  databases  as  noted  in  prior  sections  of  this  final  work  plan.  NuStats  will  not 
conduct  any  data  analysis  as  part  of  this  contract.  Data  will  be  delivered  in  a  forniat  that  will  be  most 
usable  to  NIST.  All  analysis  will  be  limited  to  statistical  calculation  of  the  data,  and  any  content  analysis 
required  to  categorize  open-ended  responses. 

A.5.1        Technical  Assistance  by  Database  Expert 

NuStats'  database  experts  will  assist  NIST  after  database  delivery  to  facilitate  NIST  developing  an 
understanding  of  the  structure,  architecture,  limitations,  and  use  of  the  database  (this  can  include  interim 
database  delivery  if  requested  by  NIST).  NuStats  will  also  deliver  a  Data  User's  Guide  as  part  of  final 
deliverable  to  NIST.  While  this  Guide  will  not  replace  the  need  for  the  on-site  consultation  by  our 
database  expert,  our  past  experience  has  shown  that  it  greatly  facilitates  client  use  of  the  database.  The 
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Guide  will  include  descriptions  of  the  data  file,  sample  tables,  data  file  codebooks,  glossary  of  terms, 
discussion  of  estimated  sampling  errors,  and  additional  background  information. 

A.6  SCHEDULE 


The  following  provides  a  schedule  of  project  milestones  and  meetings. 


Task 

Description 

Due  Date 

Days  from 
Award 

Contract  Award 

6/2 

0 

1 

Final  Work  Plan 

6/16 

13 

CATl  and  CAPl  Instrument  Drafts 

7/1 

35 

Meeting  with  NIST  /  Experts  to  Review  Drafts 

7/8 

42 

Revised  CATI  and  CAPl  Instruments  and  Focus  Guide  Protocols 

7/22 

56 

Respondent  Materials 

7/22 

56 

IRB  Approval 

8/5 

70 

NIST  IRB  Approval 

8/19 

84 

CATI  Pilot  Training 

8/21 

86 

CATI  Pilot 

8/25 

90 

CAPl  Pilot 

8/26-28 

93 

Pre-Field  Work  Meeting 

8/28-29 

94 

Finalize  Instruments  and  Procedures 

9/15 

110 

Training  CAPl 

9/17-18 

113 

0MB  Approval 

9/22 

117 

Training  CATI 

9/23-24 

119 

2 

Begin  CAPl  Field  Work 

9/23 

119 

Begin  CATI  Field  Work 

10/1 

127 

First  Data  Delivery  and  Status  Report 

10/3 

129 

First  Focus  Group  Sessions 

10/6-9 

135 

Data  Review  Meeting 

10/10 

136 

Second  Focus  Group  Sessions 

10/27-30 

156 

Second  Data  Delivery  and  Status  Report 

11/3 

160 

Third  Focus  Group  Sessions 

11/3-6 

163 

Fourth  (Last)  Focus  Group  Sessions 

11/17-20 

177 

End  CATI  Field  Work 

11/27 

184 

Third  Data  Delivery  and  Status  Report 

12/3 

190 

End  CAPl  Field  Work 

12/12 

199 

Debrief  on  Field  Work  and  Delivery  of  Summary  Reports 

12/19 

204 

Final  Field  Work  Report 

12/19 

204 

3 

Final  Data  Delivery 

12/19 

206 

Data  Users  Guide 

12/19 

206 
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Data  Collection  Methodology  for  World  Trade  Center 
Evacuation  AND  Emergency  Response:  Telephone  Interviews, 
Face-to-Face  Interviews,  Focus  Groups  and  Population 

Sampling 


1.  Background 

The  goal  of  the  National  Institute  of  Standards  and  Technology's  World  Trade  Center  Investigation  is  to 
investigate  the  building  construction,  the  materials  used,  and  the  technical  conditions  that  contributed  to 
the  outcome  of  the  World  Trade  Center  (WTC  )  disaster.  The  results  of  the  Investigation  will  serve  as  the 
basis  for  improvements  in  the  way  buildings  are  designed,  constructed,  maintained,  and  used;  improved 
tools,  guidance  for  industry  and  safety  officials;  revisions  to  codes,  standards,  and  practices;  and 
improved  public  safety.  The  primary  objectives  of  the  NIST-led  technical  investigation  of  the  WTC 
disaster  are  to: 

1 .  Determine  why  and  how  WTC  1  and  2  collapsed  following  the  initial  impacts  of  the  aircraft  and 
why  and  how  WTC  7  collapsed; 

2.  Detennine  why  the  injuries  and  fatalities  were  so  high  or  low  depending  on  location,  including  all 
technical  aspects  of  fire  protection,  occupant  behavior,  evacuation,  and  emergency  response; 

3.  Determine  what  procedures  and  practices  were  used  in  the  design,  construction,  operation,  and 
maintenance  of  WTC  1,  2,  and  7;  and 

4.  Identify,  as  specifically  as  possible,  areas  in  building  and  fire  codes,  standards,  and  practices  that 
are  still  in  use  and  warrant  revision. 

The  NIST  Investigation  Plan  can  be  found  at  http://v^c.nist.gov,  including  a  description  of  Projects  7  and 
8.  Under  Project  7,  "Occupant  Behavior,  Egress,  and  Emergency  Communications,"  first-hand  accounts 
of  the  events  of  September  11,  2001  from  inside  WTC  1,  2,  and  7  will  be  collected.  This  data  collection 
effort  will  evaluate  the  role  of  occupant  behavior  and  evacuation  technologies  and  practices  for  tall 
buildings,  including  decision-making  and  situation  awareness,  time-constrained  evacuation  strategies, 
communications,  role  of  floor  wardens  and  fire  safety  directors,  and  issues  concerning  people  with 
disabilities.  Additionally,  NIST  will  seek  specific  observations  of  fire  and  smoke  conditions  and/or 
structural  damage  from  within  the  building.  Families  of  the  victims,  who  communicated  with  loved  ones 
inside  the  Towers  before  collapse,  will  be  interviewed  to  determine  the  nature  of  the  environment  above 
the  floors  of  impact. 

The  nature  of  the  communications  between  and  among  different  groups  within  the  World  Trade  Center 
has  been  identified  as  being  a  potentially  significant  factor  in  determining  the  outcome  of  the  evacuation 
and  emergency  response.  The  project  will  investigate  the  content  and  timing  of  communications  among 
the  occupants  and  authorities  within  the  buildings,  as  well  as  people  outside  the  buildings.  The  figure 
below,  a  hypothetical  demonstration  of  the  extraordinary  flow  of  information  on  the  morning  of 
September  1 1"^,  reinforces  the  need  to  understand  the  role  of  information  transfer  in  explaining  occupant 
and  responder  actions.  In  addition  to  the  intergroup  communications,  communications  within  each  group, 
particularly  the  building  occupants,  are  potentially  important  to  understanding  the  events  of 
September  11 '^ 
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Emergency  Communications 


The  objectives  of  Project  8,  "Fire  Service  Technology  and  Guidehnes,"  are  to  build  upon  work  already 
done  by  the  Fire  Department  of  New  York  (FDNY)  and  McKinsey  &  Company  by:  ( 1 )  fully  documenting 
what  happened  during  the  response  by  the  fire  services  to  the  attacks  on  the  World  Trade  Center,  up  to  the 
time  of  collapse  of  WTC  7;  (2)  identifying  issues  that  need  to  be  addressed  in  changes  to  practice, 
standards  and  codes;  (3)  identifying  ahemative  practices  and/or  technologies  that  may  address  these 
issues;  and  (4)  identifying  R&D  needs  that  advance  the  safety  of  the  fire  service  in  responding  to  massive 
fires  in  tall  buildings.  Thus,  a  subset  of  the  emergency  responders  who  were  present  at  the  World  Trade 
Center  complex  will  be  asked  to  voluntarily  participate  in  the  face-to-face  interview  or  focus  group 
phases  of  this  project.  Only  first  responders  who  participated  in  fire  suppression,  operational,  or  search 
and  rescue  activities  prior  to  the  building  collapse  will  be  considered  for  inclusion  in  the  population  of 
face-to-face  interviewees. 

The  data  collection  will  be  conducted  by  a  yet-to-be-selected  contractor  and  is  planned  to  begin  as  soon  as 
the  necessary  pre-work  is  complete.  This  includes  preparation  of  the  telephone  interview  schedule,  face- 
to-face  interview  protocol,  focus  group  protocol,  training  of  contractor  staff,  and  approval  by  NIST  and 
the  appropriate  Institutional  Review  Board  (IRB)  to  assure  compliance  with  federal  requirements  for  the 
protection  of  human  subjects.  NIST  will  use  established  procedures  to  review  all  survey  and  interview 
questions,  data  collection  methods,  and  safeguards  for  maintaining  privacy  and  confidentiality  of  all 
instruments  before  proceeding  with  these  critical  data  collection  efforts. 

Note  that  this  paper  identifies  specific  populations  and  the  size  of  samples  to  be  included  in  the  data 
collection  effort.  The  exact  numbers  and  populations  may  be  modified  to  better  suit  the  Investigation  as 
additional  details  of  the  methodology  are  finalized  by  NIST  and  the  yet-to-be-chosen  contractor. 
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2.  Overview  of  Methodological  Approach 

A  multidisciplinary,  triangulated  approach,  including  telephone  interviews,  face-to-face  interviews,  as 
well  as  focus  group  interviews  has  been  selected.  The  multi-methodological  approach  was  selected  for 
several  reasons.  First,  multiple  methodologies  increase  confidence  in  the  conclusions  and  findings  when 
more  than  one  methodology  arrives  at  the  same  conclusions.  Second,  the  varied  objectives  of  the 
Investigation  mandate  complementary  approaches  to  accomplish  all  the  goals.  Finally,  concerns 
associated  with  the  time  latency  since  September  1 1,  2001  suggest  the  use  of  different  approaches  and 
techniques  in  order  to  increase  memory  recall  and  accuracy.  A  discussion  of  each  methodology  and 
statistical  sampling  will  follow. 

NIST  intends  to  solicit  experienced  contractors  to  perfonri  the  telephone  interviews,  face-to-face 
interviews,  and  focus  groups.  The  contractor  will  meet  or  exceed  all  Federal  requirements  regarding  the 
Common  Rule  for  the  Protection  of  Human  Subjects,'  including  Institutional  Review  Board  (IRB)  and 
NIST  approvals.  The  objective  is  to  perform  up  to  600  face-to-face  interviews  of  occupants  from  areas  of 
interest,  approximately  150  face-to-face  interviews  of  first  responders  using  selected  groups, 
approximately  800  telephone  interviews  covering  selected  floors  of  WTC  1  and  WTC  2.  Additionally, 
NIST  will  contract  for  the  conduct  of  up  to  10  focus  group  sessions  with  first  responders,  including  the 
Fire  Department  of  New  York  (FDNY),  the  New  York  City  Police  Department  (NYPD),  the  Port 
Authority  Police  Department  (PAPD),  or  other  groups  identified  as  having  operational  or  command 
authority  at  the  World  Trade  Center  on  September  11,  2001.  Finally,  up  to  five  focus  group  sessions  will 
be  conducted  with  selected  building  occupants  and  management. 

2.1  Telephone  Interview  Format 

The  first  data  collection  instrument  is  the  telephone  interview.  The  collection  mechanism  will  be  a 
computer-assisted  telephone  interview.  The  primary  goal  of  the  telephone  interview  is  to  provide 
qualitative  and  quantitative  occupant  behavior  and  egress  data  which  can  be  generalized.  A  secondary 
goal  will  be  to  provide  unique,  investigative  observations,  particular  to  the  events  at  the  World  Trade 
Center  on  September  1 1"^.  The  telephone  interview  schedule  will  be  closely  linked  to  the  evacuation 
experience  of  the  occupants. 

The  questions  will  flow  in  a  logical  order  in  relation  to  the  chronology  of  the  events,  as  suggested  in  the 
literature."'^  Significant  topic  areas  proposed  for  the  telephone  interview  include,  but  are  not  limited  to: 
occupant  demographics  and  inherent  traits,  chronology  of  occupant  activity,  observations  and  perceptions 
during  evacuation,  and  environmental,  social,  psychological,  physiological,  information,  frequency,  and 
source  attributes.  As  the  precise  content  of  the  telephone  interviews,  face-to-face  interviews,  and  focus 
groups  has  not  yet  been  established,  these  factors  are  subject  to  review  and  change. 

NIST  will  follow  standard  telephone  interview  construction  techniques.'*  These  techniques  suggest  that 
the  project  team  identify  the  scope  and  objectives  of  the  telephone  interview  schedule.  The  question  type 
and  format  which  best  accomplishes  the  scope  and  objectives  will  then  be  selected.  The  first  draft  of  the 
telephone  interview  schedule  will  then  be  reviewed  and  revised.  Cognitive  and  pilot  testing  of  the 
telephone  interview  with  the  informants  will  then  occur.  After  further  revision,  the  procedures  to 
administer  the  study  will  be  specified. 

The  persons  who  were  in  WTC  1  and  2  immediately  prior  to  the  first  aircraft  impact  on  September  1 1 , 
2001,  will  constitute  the  population  to  be  sampled  in  this  study  segment,  hereafter,  to  be  referred  to  as  the 
"selected  floors  study."  The  sampling  plan  of  the  September  1 1,  2001  occupants  is  a  multi-stage 
statistically  representative  sample  with  two  stages.  The  first  stage  will  stratify  floors  by  area,  population, 
and  number  of  tenants.  The  second  stage  will  select  occupants  from  the  floors  selected  in  the  first  stage. 

Stratifying  the  population.  Stage  one  is  an  area  sample  of  floors.  WTC  1  and  2  will  be  segmented  into 
three  zones  each  according  locations  of  the  sky  lobbies.  These  zones  will  approximately  represent  the  top 
(TS'"  -  1 10""  floors),  middle  (44*  to  78*  floors),  and  lower  (ground  to  43"*  floor)  thirds  of  WTC  1  and  2. 
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This  will  result  in  a  total  of  six  building  zones  from  WTC  1  and  2.  Each  zone  will  then  be  further 
stratified  by  tenant  number  and  occupant  density.  Tenant  number  will  include  two  levels:  single  tenant 
floors  and  multi-tenant  floors.  Occupant  density  will  include  three  levels:  low,  medium,  and  high  density. 
Definitions  of  low,  medium,  and  high  density  will  be  formed  after  the  total  building  population  is 
identified.  Thus,  each  zone  will  contain  six  floors,  if  possible.  The  second  stage  is  a  random  sample 
without  replacement  of  occupants  from  the  floors  selected  in  the  first  stage. 

Enumerating  the  population.  A  population  list  of  all  the  people  in  each  of  these  6  building  strata 
immediately  prior  to  first  impact  on  September  11,  2001  will  then  be  enumerated.  It  is  estimated  that  a 
total  of  10,000  -  14,000  people  were  inside  WTC  1  and  2  at  the  time  of  the  first  impact,^  and  there  are 
unsubstantiated  accounts  of  between  4000  and  5000  persons  in  WTC  7  on  the  morning  of  September  1 1"^. 
For  the  purposes  of  sampling  and  estimation  of  the  number  of  telephone  interviews,  face-to-face 
interviews,  and  focus  groups,  this  project  will  assume  an  initial  population  of  18,500  occupants.  NIST 
will  provide  to  the  contractor  an  enumeration  of  persons  on  the  selected  floors. 

Selecting  the  sample.  The  sampling  plan  for  the  selected  floors  study  will  be  constructed  such  that  a  total 
of  800  telephone  interviews  are  obtained  from  people  included  in  the  study.  It  is  assumed  that  there  will 
be  an  approximate  30  percent  participation  rate  among  those  asked  to  participate.  A  total  of  800  telephone 
interviews  is  selected  since  this  is  twice  the  estimated  number  required  to  obtain  a  0.05  level  of  statistical 
significance.^  The  contractor  will  make  every  reasonable  effort  to  increase  participation  above  30 
percent. 

Data  collection.  This  segment  of  the  study  will  use  a  computer  assisted  telephone  interview  to  obtain  data 
from  those  who  choose  to  participate  in  the  study. 

2.2  Face-to-face  interview  Format 

The  objective  of  the  face-to-face  interview  segment  is  to  gather  first-hand  accounts  and  observations  of 
the  activities  and  events  inside  the  buildings  on  the  morning  of  September  1 1'''.  This  approach  will 
identify  unknown  information,  evaluate  technical  hypotheses,  and  explore  conscious  and  subconscious 
motivations  for  occupant  and  responder  behaviors,  while  allowing  for  comparisons  to  the  telephone 
interview  data.  There  will  be  no  verbatim  record  of  the  face-to-face  interviews,  other  than  random 
selections  for  quality  control  purposes.  It  is  estimated  that  the  average  face-to-face  interview  will  last 
approximately  two  hours,  with  some  lasting  significantly  longer. 

The  proposed  methodology  for  the  face-to-face  interviews  is  a  synthesis  of  the  Behavioral  Sequence 
Interview  Technique  (BSIT)  originally  developed  by  Keating  and  Loftus,'  and  the  Cognitive  Interviewing 
Method  (CIM),  originally  developed  by  Fisher^  and  Geiselman.^  These  two  interviewing  methodologies 
were  developed  with  the  purpose  of  assisting  persons  in  retrieving  more  comprehensive  and  accurate 
memories  of  incidents,  and  sharing  important  attributes.  Both  approaches  begin  by  allowing  the  informant 
to  retell  an  unimpeded  account  without  interruption  from  the  interviewer,  and  both  initially  employ  a 
chronological  retelling  of  information.  However,  BSIT  was  designed  to  yield  a  database  of  quahtative 
information  that  could  be  subjected  to  systematic  analysis  and  consolidation,  while  CIM  was  designed  to 
facilitate  investigative  interviews.  Since  the  Investigation  is  pursuing  both  goals  (i.e.,  creation  of  a 
database  of  evacuation-related  behaviors  and  an  investigatory  attempt  to  capture  information  relevant  to 
outcomes),  the  proposed  methodology  combines  these  two  approaches. 

Cognitive  interviewing  has  been  the  subject  of  many  empirical  investigations.  Fisher,  et  al.'"  summarized 
these  findings,  demonstrating  that  the  methodology  significantly  increases  the  amount  of  information 
recalled  without  affecting  rate  of  errors.  Interviewing  a  large  number  of  informants  will  allow 
coiToboration  of  infonnation,  thereby  compensating  for  the  likely  increase  in  the  absolute  number  of 
errors.  Accordingly,  it  is  likely  that  this  approach  will  be  productive  in  achieving  a  holistic  view  of  the 
building  evacuations. 
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The  face-to-face  interview  methodology,  hereinafter  referred  to  as  the  "areas  of  interest  study,"  will 
involve  face-to-face  interviews  of  occupants  and  first  responders  who  may  have  observed  (knowingly  or 
unknowingly)  events  important  to  completion  of  the  objectives  of  the  hivestigation. 

EnumeratinR  the  population.  The  population  will  include  the  entire  occupant,  management,  and  first 
responder  population  of  World  Trade  Center  WTC  1,  2,  and  7. 

Selecting  the  sample.  The  areas  of  interest  sample  will  identify  individuals  using  the  snowball  quota 
sample  approach  whose  constituency  may  resemble  individuals  selected  for  the  "Specialized  Groups 
Study"  sampling  methodology  (see  below).  A  snowball  quota  sample  approach  asks  individuals  for  the 
names  of  other  people  who  may  meet  the  selection  criteria  for  the  study.  These  people  are  subsequently 
contacted  and  asked  the  same  question.  The  process  continues  until  the  quota  has  been  reached.  The  goal 
is  to  perfonn  approximately  600  face-to-face  interviews  with  occupants,  30  face-to-face  interviews  with 
family  members  who  communicated  with  victims  inside  the  building  during  the  event,  and  150  first 
responders.  The  150  first  responders  will  be  divided  among  the  Fire  Department  of  New  York 
(firefighters,  company  officers,  and  operational  command  officers).  Port  Authority  Police  Department, 
New  York  Police  Department,  and  other  responsible  parties.  Additional  individuals  may  be  randomly 
selected  from  strata  previously  defined  in  the  whole  buildings  study  in  order  to  compare  the  face-to-face 
interview  results  with  the  results  of  the  telephone  interviews. 

Data  Collection.  The  face-to-face  interviews  will  follow  a  four  step  technique,  including  unimpeded, 
open-ended  narrative,  a  structured  narrative,  technical  probes,  and  closed-ended  questions.  Each  step  is 
described  more  fully  below. 

Step  1:  Unimpeded  open-ended  narrative  account.  Both  BSIT  and  CIM  begin  the  process  by  asking 
the  participant  to  chronologically  recount  his  or  her  "story."  The  proposed  starting  point  is  when  it 
became  apparent  that  something  unusual  had  occurred  on  the  morning  of  September  1 1 ,  200 1 .  The 
proposed  ending  point  is  when  the  participant  feels  that  he  or  she  reached  a  location  where  they  feh  safe 
(or,  alternatively,  when  he  or  she  successfully  reached  the  exterior  of  the  building).  Researchers  and 
practitioners  involved  with  cognitive  interviewing  believe  that  starting  the  face-to-face  interviews  in  this 
manner  both  improves  recall  and  helps  build  rapport  between  the  participant  and  the  interviewer.  Fisher  et 
al.'^  also  noted  that  asking  questions  may  interfere  with  recall  because  a  participant  must  divide  his  or  her 
mental  resources  between  recall  and  listening  to  the  interviewer's  questions. 

During  the  open-ended  narrative  account,  the  interviewer  can  record  notable  information  that  can  be  used 
for  the  probing  phase  conducted  later.  For  example,  the  participant  might  briefly  mention  an  odd  odor  to 
which  the  interviewer  will  want  to  return  to  determine  whether  the  smell  might  have  been  that  of  jet  fuel, 
smoke,  or  of  some  other  origin  as  yet  unknown. 

Step  2:  Structured  narrative  account.  After  participants  complete  their  stories,  interviewers  will  prompt 
them  to  go  through  the  story  again,  but  this  time  they  will  work  cooperatively  with  the  interviewer  to 
record  entries  into  a  table.  This  approach  is  employed  by  BSIT  for  three  primary  reasons:  (1)  to  yield  a 
structured  account  that  can  be  entered  into  a  database  without  further  processing;  (2)  to  avoid  the  biasing 
effects  of  having  interviewers  ask  specific  questions;  and,  (3)  to  enhance  the  effort  at  recall  put  forward 
by  participants  by  encouraging  their  active  collaborative  participation,  an  advantage  to  open-ended 
formats  as  noted  by  Fisher,  et  al.'" 

Each  row  of  the  table  will  represent  a  single  action  in  a  sentential  format,  meaning  that  each  action  is 
expressed  as  a  grammatical  sentence.  The  approach  is  used  based  on  the  hypothesis  that  people  encode 
episodic  memories  in  a  manner  consistent  with  this  format,  thus  facilitating  both  recall  and  data  entry. 
Each  column  of  the  table  represents  three  essential  components  of  actions:  a  cue,  an  action,  and  the  reason 
for  taking  that  action.  Cues  can  be  either  external  (e.g.,  signs  of  a  fire,  someone  saying  something)  or 
internal  (e.g.,  remembering  about  another  means  of  escape.)  Actions  are  expressed  using  specific  action 
verbs  (i.e.,  "ran"  instead  of  "wenf )  and  may  include  artifacts  (e.g.,  a  fire  extinguisher)  used  by  the 
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informant.  Reasons  are  the  intentional,  goal-directed  base  for  the  action.  The  interviewer  will  encourage 
the  participant  to  use  their  own  words  to  the  greatest  extent  possible. 

A  hypothetical  example  of  actions  recorded  in  this  manner  is: 


Cue 

Action 

Reason 

1  heard  but  couldn't  see 

someone  yell  "I've  found  a 
clear  path" 

So  1  stumbled  in  the  dark  towards 
where  1  thought  the  voice 
came 

So  that  1  could  find  a  way  to 
escape 

My  path  was  blocked  by  debris 

So  1  called  out  to  whoever  yelled, 
"I'm  near  the  reception  area. 
Where  are  you?" 

To  try  to  get  a  better  idea 
about  where  the  person 
was 

Table  1:  Example  Tabular  Face-to-face  interview  Data  Entry 


Experimental  findings  in  psychological  research  on  memory' '  suggest  that  when  people  perform  actions, 
their  abilities  to  verbally  recall  those  actions  are  significantly  improved.  Script  theory'"  suggests  that 
people  naturally  organize  their  knowledge  of  actions  using  narrative  sequences  of  actions  structured 
around  their  pursuit  of  goals.  However,  gaps  in  the  narrative  are  anticipated,  especially  given  the  long 
period  of  time  that  will  have  elapsed  between  the  event  and  the  interview.  Interviewers  will  assist  the 
participants  to  fill  in  these  gaps  by  asking  them  to  recall  events  in  reverse  order,  an  approach  used  in 
CIM.  Inter\'iewers  will,  however,  encourage  participants  to  report  only  those  memories  about  which  they 
are  confident  really  occurred  to  them. 

Step  3:  Probing  for  specific  information.  After  completing  the  structured  narrative  account, 
interviewers  will  ask  specific  open-ended  questions  (probes)  intended  to  provide  specific  information  of 
particular  value  to  the  investigation.  While  some  of  this  information  is  likely  to  be  part  of  the  structured 
narrative  account,  participants  may  be  able  to  recall  other  valuable  information  as  well. 

Interviewers  may  use  "context  reinstatement"  from  CIM  to  improve  recall  of  important  information, 
because  laboratory  experiments  have  demonstrated  that  contextual  cues  enhance  recall  of  related 
information.  Fisher  et  al.  explain  that  context  reinstatement  may  enhance  recall  because  people  use 
muhisensory  coding  of  events.  Using  this  mnemonic  method,  interviewers  will  ask  participants  to 
"mentally  recreate  the  external  environment,  and  their  affective,  physiological,  cognitive,  and  emotional 
states  that  existed  at  the  time  of  original  event." 

Depending  on  the  population,  probes  may  be  used  to  try  to  elicit  information  including,  but  not  limited  to: 

•  Location  of  the  informant  at  the  time  of  certain  marker  events  (e.g.,  location  in  WTC  1  when 
WTC  2  collapsed) 

•  Fire  conditions  (e.g.,  fire  and  smoke); 

•  Other  cues  of  interest  (e.g.,  the  smell  of  jet  fuel); 

•  Presence  and  activities  of  persons  with  disabilities; 

•  Use  of  elevators  by  self  or  others;  and, 

•  Knowledge  of  any  obstacles  to  their  progress  while  using  the  stairs. 

Because  information  about  many  of  these  areas  of  concern  requires  precise  responses,  questions  for  open- 
ended  probes  will  be  developed  collaboratively  between  the  contractor  and  NIST.  Responses  to  probes 
may  be  recorded  using  standardized  formats  where  feasible.  For  example,  all  participants  who  observed 
smoke  may  be  asked  to  estimate  the  smoke  density  using  an  encodable  scale,  such  as  visibility  distance. 

Step  4:  Close-ended  telephone  interview  items.  Participants  will  be  asked  to  complete  at  least  some 
items  from  the  telephone  interview.  These  items  are  likely  to  be  placed  at  the  end  of  the  face-to-face 
interview  session  to  avoid  biasing  open-ended  responses.  The  purpose  of  administering  telephone 
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interview  items  is  to  determine  whether  there  are  systematic  differences  among  the  representative  samples 
used  for  telephone  interviews,  face-to-face  interviews,  and  focus  groups. 

Quality  Control  for  Face-to-face  interviews.  To  ensure  that  mtcrviewcrs  are  complying  with  the  facc-to- 
face  interview  techniques  and  administrative  requirements,  this  project  is  proposing  that  interviews  be 
videotaped,  with  the  following  provisions:  the  participant  signs  a  form  accurately  describing  the  reasons 
for  and  retention  of  videotapes  and  granting  pennission  for  the  face-to-face  interview  to  be  videotaped; 
the  contractor  will  destroy  the  video  record  at  the  conclusion  of  the  investigation;  review  for  quality 
control  will  be  the  only  purpose  for  videotaping  the  face-to-face  interview;  no  transcriptions  will  be  made. 
NIST  and  the  contractor  will  periodically  review  videotapes  to  ensure  that  interviewers  precisely  follow 
the  protocol  and  confonn  to  administrative  requirements. 

2.3  Focus  Groups 

WiUiams'^^  reports  that  in  a  group  setting,  people  provide  cues  that  evoke  memories  in  others,  and  that 
social  pressures  mediate  against  reporting  misrepresentations  of  what  they  recall.  Thus,  the  goal  of  the 
focus  group  interviews  is  to  elicit  accurate  group  representations  of  specific  events  or  themes.  Two 
distinct  populations  will  voluntarily  participate  in  the  focus  groups:  occupants  and  first  responders.  The 
first  set  of  focus  group  interviews  will  be  the  occupant  sample.  Distinct  categories  of  people  will  be 
selected  for  inclusion  in  this  study,  hereafter  referred  to  as  the  "specialized  groups  study."  The  objective 
of  this  study  will  be  to  capture  the  experience  of  people  in  unique  places  in  WTC  1,  2,  and  7.  These 
groups  will  be  defined  by  the  NIST  Investigation  team.  Every  effort  will  be  made  to  include  no  less  than  5 
people  in  each  of  these  categories  in  this  study,  with  10  people  constituting  the  preferred  focus  group  size. 
NIST  anticipates  conducting  approximately  5  occupant  focus  groups. 

First  responders  will  constitute  a  second  set  of  focus  group  interviews.  The  set  of  first  responders  will 
include  FDNY,  NYPD,  PAPD,  and  other  groups  identified  as  having  operational  or  command  authority  at 
the  World  Trade  Center  on  September  1 1''\  The  focus  group  size  will  be  determined  as  an  operating  unit 
size,  if  applicable.  An  operating  unit  may  be  a  Fire  Department  company,  for  example.  This  project 
proposes  10  focus  groups,  each  containing  5  people. 

Sample  selection.  The  people  selected  for  inclusion  in  this  study  will  be  selected  using  non-probability 
sampling  procedures.  The  contractor  will  use  a  snowball  quota  sample.'''"'"''  Respondents  contacted  or 
face-to-face  interviewed  for  other  reasons  will  be  asked  to  provide  the  names  and  contact  information  for 
people  they  know  in  each  of  the  categories  in  the  specialized  groups  study.  Names  will  be  collected  by  the 
contractor  until  at  least  5  people  in  each  category  have  agreed  to  participate  in  an  occupant  focus  group, 
with  a  preference  for  1 0  people.  The  same  process  will  occur  for  selection  of  the  first  responder  samples, 
with  a  preference  for  inclusion  of  entire  operating  units  (about  5  people  per  unit). 

Data  collection.  Focus  groups  will  be  conducted  with  the  members  of  each  group  selected  for  inclusion  in 
each  of  the  specialized  categories  included  in  this  study  segment.  The  data  collected  in  this  study  will 
produce  qualitative  and  detailed  narrative  accounts  of  the  experiences  of  each  category  of  people.  The 
focus  group  discussion  will  be  moderated  by  a  trained  and  experienced  contractor. 

3.  Database 

The  contractor  will  provide  to  NIST  at  the  conclusion  of  the  project  a  database  of  encoded  survey  results. 
Each  telephone  interview,  face-to-face  interview  and  focus  group  will  result  in  an  encoded  table  of  resuhs 
which  can  be  analyzed  using  standard  data  analysis  techniques,  such  as  averages,  multivariate  regression, 
and  statistical  significance.  The  specific  identity  of  the  encoding  variables  will  be  generated  jointly  by 
the  contractor  and  NIST  and  is  subsequent  to  the  actual  content  of  the  survey  instruments,  which  will  also 
be  developed  by  the  contractor,  subject  to  input  and  approval  from  NIST.  The  number  of  encoding 
variables  is  anticipated  to  be  less  than  75.  This  database  will  need  to  be  consistent  with  an  analysis  of 
third-party  and  media  accounts  which  NIST  will  generate  and  code  independently  of  any  contractors. 
Analysis  of  all  data  and  any  conclusions  derived  therein  will  be  the  sole  responsibility  of  NIST. 
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However,  it  is  anticipated  that  a  database  expert  from  the  survey  contractor  will  assist  NIST  after  database 
delivery  in  developing  an  understanding  of  the  structure,  architecture,  limitations,  and  use  of  the  database. 

4.  Latency  and  Accuracy  of  Recall 

The  accuracy  of  participants'  memories  of  events  is  a  consideration,  especially  given  the  period  of  time 
that  will  have  elapsed  between  the  September  1 1  attacks  and  the  data  gathering  activities.  Empirical 
investigations  reveal  that  greater  amounts  of  information  are  recalled  using  CI  methods  without 
increasing  the  rate  of  errors.  For  example,  as  compared  to  traditional  epidemiological  interviews,  Fisher  et 
al.  were  able,  with  CI  methods,  to  elicit  many  more  responses  and  more  precise  responses  from  people 
asked  to  recall  daily  physical  activities  from  35  years  earlier. 

NIST  will  address  latency  in  two  ways.  First,  multiple  participants  who  would  have  experienced  similar 
situations  will  be  used  to  corroborate  as  much  of  the  infonnation  as  possible.  Thus,  information  that 
cannot  be  reconciled  with  other  evidence  may  be  discounted.  Second,  the  proposed  investigative 
approaches  are  expected  to  increase  the  accuracy  of  the  data  collected.  In  a  review  of  research,  Pezdek 
and  Taylor'^  concluded  that  people  retain  fairly  accurate  memories  of  directly  experienced  events.  They 
hypothesized  that  participation  in  events  leads  to  coherent  well-structured  narrative  memories.  Because 
NIST  will  only  be  asking  about  directly  experienced  events,  and  will  be  asking  participants  to  recall 
events  in  a  manner  compatible  with  their  naturally  occurring  internal  representations,  the  accuracy  of 
recall  should  be  acceptable. 

5.  Protection  of  Human  Subjects 

This  data  gathering  effort  will  ensure  that  all  precautions  required  by  the  Common  Rule  for  the  Protection 
of  Human  Subjects  are  met  or  exceeded  by  the  contractor.  Participation  in  any  part  of  this  project  by  any 
person  will  be  strictly  voluntary.  Interviewers  will  be  trained  to  establish  a  rapport  with  participants 
based  on  a  compassionate  interest  in  their  story  and  will  ensure  participants  that  information  provided  will 
be  of  value  in  preventing  casualties  in  future  building  emergencies.  During  the  briefing,  interviewers  will 
provide  information  to  participants  about  where  and  how  to  receive  counseling  without  charge,  and  that 
participants  may  stop  the  interview  at  any  time  without  explanation.  Interviewers  will  also  be  trained  to 
recognize  signs  of  post-traumatic  stress.  Similar  services  will  be  offered  to  participants  of  focus  groups 
and  to  people  taking  the  telephone  interview.  Finally,  the  contractor  will  take  the  necessary  precautions 
to  ensure  the  safety  of  contract  employees  administering,  collecting,  or  otherwise  involved  in  this  data 
collection  effort. 

6.  Additional  Data  Collection 

The  scenario  may  arise  where  an  individual  critical  to  developing  an  understanding  of  the  events  of 
September  11,  2001,  may  be  unavailable  or  unidentified  during  the  period  of  performance  of  the 
Contractor.  Thus,  NIST  will  obtain  NIST  IRB  approval  to  conduct  a  limited  number  of  face-to-face 
interviews  with  individuals  deemed  by  NIST  likely  to  contribute  significantly  to  the  outcome  of  the 
Investigation.  The  scope,  objectives,  and  procedures  used  in  the  additional  data  collection  will  be  similar 
to  the  scope,  objectives,  and  procedures  used  by  the  contractor.  Telephone  interviews  and  focus  groups 
will  not  be  conducted  in  this  additional  data  collection  effort.  It  is  anticipated  that  the  number  of  face-to- 
face  interviews  conducted  by  NIST  will  be  less  than  10%  of  the  number  of  face-to-face  interviews 
conducted  by  the  contractor.  The  contractor  will  incur  no  duties  or  obligations  related  to  the  additional 
data  collection. 

7.  Conclusions  and  Summary 

Table  1  summarizes  the  NIST  Investigation  survey  method.  NIST  proposes  a  triangulated, 
multidisciplinary  survey  methodology  to  analyze  and  document  the  events  of  September  11,  2001,  at  the 
WTC  1,  2,  and  7.  The  three  strategies  include  telephone  interviews,  face-to-face  interviews,  and  focus 
groups.  The  triangulated  approach  was  selected  in  order  to  increase  confidence  in  the  conclusions. 
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complete  dual  objectives  of  generalization  and  investigation,  and  increase  memory  recall  and  accuracy. 
The  methodology  and  enumeration  are  summarized  below. 

The  data  collection  will  be  conducted  by  a  yet-to-be-selected  contractor  and  is  planned  to  begin  as  soon  as 
the  necessary  pre-work  is  complete.  This  includes  preparation  of  the  telephone  interview  schedule,  facc- 
to-face  interview  protocol,  focus  group  protocol,  training  of  contractor  staff,  and  approval  by  NIST  and 
the  appropriate  Institutional  Review  Board  (IRB)  to  assure  compliance  with  federal  requirements  for  the 
protection  of  human  subjects.  NIST  will  use  established  procedures  to  review  all  survey  and  face-to-facc 
interview  questions,  data  collection  methods,  and  safeguards  for  maintaining  privacy  and  confidentiality 
of  all  instruments  before  proceeding  with  these  critical  data  collection  efforts.  As  additional  details  of  the 
survey  methodology  are  finalized,  populations  to  be  included  in  this  project  may  be  modified. 

The  telephone  interview  approach  is  described  as  the  selected  floors  study.  WTC  1  and  2  will  be  divided 
into  three  zones,  low,  medium,  and  high.  The  zones  will  be  further  stratified  into  combinations  of  single- 
and  multi-tenant  floors,  as  well  as  floors  with  low,  medium,  and  high  occupant  densities  at  the  time  of  the 
first  aircraft  impact.  This  is  anticipated  to  resuh  in  a  30  percent  response  rate,  yielding  approximately  800 
respondents.  Eight  hundred  respondents  represent  a  safety  factor  of  two  for  the  population  necessary  to 
achieve  0.05  level  of  statistical  significance. 

The  areas  of  interest  study  will  be  conducted  with  face-to-face  interviews  of  up  to  600  people.  The 
potential  respondents  will  include: 

•  up  to  200  people  near  floors  of  impact, 

•  up  to  150  floor  wardens,  fire  safety  directors  and  persons  with  responsibility, 

•  up  to  100  people  in  elevators  or  lobbies, 

•  up  to  100  people  from  WTC  7, 

•  up  to  30  family  members  of  victims  who  called  out  of  the  towers,  and 

•  up  to  20  people  with  disabilities. 

The  Behavioral  Sequence  Interview  Technique  and  Cognitive  Interview  Method  will  be  combined  in  the 
face-to-face  interview  sessions.  This  approach  will  maximize  the  investigative  return  in  order  to  identify 
unknown  information,  evaluate  technical  hypotheses,  and  explore  conscious  and  subconscious 
motivations  for  occupant  and  responder  behaviors,  while  allowing  for  comparisons  to  the  telephone 
interview  data.  NIST  will  also  conduct  face-to-face  interviews  with  members  of  the  Fire  Department  of 
New  York,  Port  Authority  Police  Department,  New  York  Police  Department,  and  others  having 
operational  responsibilities.  This  approach  will  face-to-face  interview  approximately  150  people,  with  the 
population  being  stratified  among  firefighters,  company  officers,  and  operational  command  officers. 

The  third  approach  will  employ  focus  groups.  NIST  anticipates  creating  five  focus  groups  of  building 
occupants  with  approximately  10  people  per  group.  The  population  will  be  generated  using  the  snowball 
quota  sample  approach.  Additionally,  NIST  will  create  approximately  10  focus  groups  with  first 
responders,  with  each  focus  group  containing  approximately  five  individuals.  The  population  will  be 
generated  using  the  snowball  quota  sample  approach. 
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Method 

Intended 
Number  of 
Respondents 

Intended 
Response 
Rate 

Sampling 
Strategy 

Population 

Telephone 
Questionnaire 

Selected  Floors 
(Occupants) 

800 

30% 

Statistically 
Representative 

Floors 

WTC  1  and  2 

o 

Areas  of  Interest 
(Occupants) 

570 

N/A 

Snowball  Quota 
and  Randomly 
Selected 

WTC  1,2, 
and  7 

Faee  Int 

(Families) 

30 

N/A 

Snowball  Quota 

WTC  1  and  2 

Face  to 

Areas  of  Interest 

150 

N/A 

Snowball  Quota 

FDNY, 
NYPD, 
PAPD,  others 

Focus  Groups 

Specialized 

Groups 
(Occupants) 

50 

N/A 

Snowball  Quota 

WTC  1,2, 
and  7 

Specialized 
Groups  (First 
Responders) 

50 

N/A 

Snowball  Quota 

FDNY, 
NYPD, 
PAPD,  others 

Table  2:  Summary  of  Methods 
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